I agree that the Eden story is not metaphorical. But neither do I think it's about material creation. I follow the research of Dr. John Walton who has made a convincing case that Genesis 1 and 2 are about how God ordered the world to function, not about how they came about. It's a fascinating approach, and I think it has a lot of merit. So, Adam and Eve are historical people who represent all humanity. The Garden of Eden was a literal place, but it functioned as a symbol of the temple—where God would meet with his priest and priestess and reveal Himself to them.

As far as taking Jesus literally, it's obvious that Jesus speaks in many literary forms (parable, symbolism, simile, metaphor, hyperbole, etc.) and we are often not supposed to take him literally. But I believe that the genealogy of Jesus does literally and historically trace back to a historical Adam.

Therefore, your question ("So at what point in the genealogy do the stories of the bible/genesis in particular stop being metaphorical and start becoming historical?") is the wrong question. The genealogy was never metaphorical. We can talk more as you wish.

It's silly to take the Eden story metaphorically. Jesus/the gospels are pretty much always taken literally (otherwise, as according to 1 cornthians 15.you have believed in vain), but the genealogy of jesus supposedly goes back to adam. So at what point in the geneology do the stories of the bible/genesis in particular stop being metaphorical and start becoming historical?