We consider the problem of the hydrogen atom interacting with a circularly polarized microwave field, modeled as a perturbed Kepler problem. A remarkable feature of this system is that the electron can follow what we term erratic orbits before ionizing. In an erratic orbit the electron makes multiple large distance excursions from the nucleus with each excursion being followed by a close approach to the nucleus, where the interaction is large. Here we are interested in the mechanisms that explain this observation. We find that the manifolds associated with certain hyperbolic periodic orbits may play an important role, despite the fact that, in some respects, the dynamics is almost Keplerian. A study of some relevant invariant objects is carried out for different system parameters. The consequences of our findings for ionization of an electron by the external field are also discussed.