Great insight! The environment will certainly shape — and as you describe, restrict — the progression of the development and, of course, what is considered “normal” or desirable. For certain development (e.g., language, attachment) psychologists have theorized a “window of opportunity” which, if missed, results in permanent deficits. For other aspects, early deficiencies can be made up later within an enriching environment. Much depends on the specific task, the level of deprivation and the age or point at which the environment is enriched.
Thanks for reading and commenting!