AngularJS HTTP call takes two minutes while the Rails action only takes 20 seconds, how can I debug this?

I have an AngularJS app and there's one page in my application, only one, that is taking 2 minutes to load. It is loading a bit of data, but the data itself is only 700KB and I benchmarked the entire rails action starting from the beginning until right before the render and it only takes 15-20 seconds. But when I look at the actual network call, or I put a timer before the angular http post call and then one in the success, they both show the call taking almost 2 minutes. I can't figure out what's going on between the render and the success on angular that would be causing this extreme time difference. Does anyone know how I could further debug this or possibly know what could be causing this?

The rails action just does a couple big database calls, all optimized, then does some work on the data, then the data (which is already JSONified with to_json) is rendered out.

Rails action ends with Completed 200 OK in 20458ms (Views: 913.8ms | ActiveRecord: 139.6ms)