It is an allusion to Samuel Taylor Coleridge's poem The Rime of the Ancient Mariner (1798) -- which, in part, inspired Mary Shelley's Frankenstein. In the poem, an albatross starts to follow a ship — being followed by an albatross was generally considered an omen of good luck. However, the mariner shoots the albatross with a crossbow, which is regarded as an act that will curse the ship (which indeed suffers terrible mishaps). To punish him, his companions induce him to wear the dead albatross around his neck indefinitely (until they all die from the curse). Thus the albatross can be both an omen of good or bad luck, as well as a metaphor for a burden to be carried as penance.
The symbolism used in the Coleridge poem is its highlight. For example:
Ah ! well a-day ! what evil looks
Had I from old and young !
Instead of the cross, the Albatross
About my neck was hung.
An "albatross around one's neck" means a burden which some unfortunate person has to carry.