Entity browser itself does not need information about entity type being selected. It can actually select entities of different type. However, some of the display plugins need to know which entity type they are operating with. Display plugins that do not need this info will ignore this configuration value.