Skip to content

Conversation

@miceno
Copy link
Owner

@miceno miceno commented Jan 3, 2025

Not all HTTP servers follow the HTTP protocol standard that requires headers to be CR+LF terminated.
Some servers do not send CRLF at the end of lines but just LF, some others produce a mix of headers ending in CRLF or LF.

The response from such servers is not supported by this library, and thus the library fails to product a reliable response.

HTTP standard allows for such behaviour (see Message Parsing Robustness) and recommends reading lines till the LF and discard any CR.

This PR tries to mitigate this issue. It was detected while getting responses from the private API of ZTE MF65 4G MIFI router. The server responses didn't include a CR+LF on every header, only some of them included the CR+LF but others include only LF.

Here it is the output of curl using --trace - option on one of those routers:

$ curl -0 --trace - --header @headers-get.txt "http://192.168.0.1/goform/goform_get_cmd_process?isTest=false&multi_data=1&cmd=battery_value";
...
<= Recv header, 16 bytes (0x10)
0000: 48 54 54 50 2f 31 2e 30 20 32 30 30 20 4f 4b 0a HTTP/1.0 200 OK.
<= Recv header, 22 bytes (0x16)
0000: 53 65 72 76 65 72 3a 20 47 6f 41 68 65 61 64 2d Server: GoAhead-
0010: 57 65 62 73 0d 0a                               Webs..
<= Recv header, 17 bytes (0x11)
0000: 50 72 61 67 6d 61 3a 20 6e 6f 2d 63 61 63 68 65 Pragma: no-cache
0010: 0a                                              .
<= Recv header, 24 bytes (0x18)
0000: 43 61 63 68 65 2d 63 6f 6e 74 72 6f 6c 3a 20 6e Cache-control: n
0010: 6f 2d 63 61 63 68 65 0a                         o-cache.
<= Recv header, 24 bytes (0x18)
0000: 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 74 65 Content-Type: te
0010: 78 74 2f 68 74 6d 6c 0a                         xt/html.
<= Recv header, 1 bytes (0x1)
0000: 0a                                              .
<= Recv data, 22 bytes (0x16)
0000: 7b 22 62 61 74 74 65 72 79 5f 76 61 6c 75 65 22 {"battery_value"
0010: 3a 22 35 33 22 7d                               :"53"}
...

Cache-control header ends in LF (0xa) while Server header ens in CRLF (0xd 0xa).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant