Can Bigger Always Be Better?