How do you perform HTTP requests from App Engine to internal services?

Question

I have an App Engine Standard (1st Gen) Python app, set up with serverless VPC access. I use requests 2.3 (I have issues with later versions on GAE) to do HTTP requests.

When I try doing an HTTP request to an internal IP address (10.x), it refuses to connect. However, serverless VPC access is working properly: I tested connecting to the same IP address on the same port using a non-HTTP client library (e.g. redis) from App Engine, and this works fine. It's just HTTP requests that fail.

I suspect the URL Fetch service is trying to do the HTTP requests, and fails on internal addresses. If so, is there a way to use requests without triggering the URL Fetch service? Or do you have any other clue what might be going wrong?

As far as I can tell from those links, you can only force the use of sockets globally by setting GAE_USE_SOCKETS_HTTPLIB. However, I'm experiencing more problems than usual by doing so; is there a way to use sockets just for a few specific calls? — Remko
– Remko, Commented Apr 8, 2020 at 19:29

Remko · Accepted Answer · 2020-04-09 20:25:40Z

1

I think I found the answer by looking at the SDK source code:

from python_std_lib import httplib

gets you an HTTPLib that uses sockets, without having to override the global httplib (causing sockets to be used everywhere).

answered Apr 9, 2020 at 20:25

Remko

8776 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How do you perform HTTP requests from App Engine to internal services?

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related