API sometimes returns `{"error": 1}`

Server
1

Why does the UrBackup API work in only some cases?

We are attempting to use the UrBackup API on an Infscape appliance. Please find debugging information below.

I have two machines: machine A (Debian 11) and machine B (macOS 12.5.1).

On both machines, the http.HTTPConnection code from the official Python API client works:

>>> h = http.HTTPConnection('infscape-test.cyberfusion.nl', 80, timeout=1)
>>> h.request('POST', '/x?a=add_client', 'clientname=test2&ses=x', {'Accept': 'application/json', 'Content-Type': 'application/json; charset=UTF-8'})
>>> h.getresponse()
<http.client.HTTPResponse object at 0x7fd36906b250>
>>> _.read()
b'{\n"already_exists": true,\n"ok": true\n}\n'

On both machines, executing the same request with requests fails:

>>> requests.post("http://infscape-test.cyberfusion.nl:80/x?a=add_client", data='clientname=test2&ses=x', headers={'Accept': 'application/json', 'Content-Type': 'application/json; charset=UTF-8'}).json()
{'error': 1}

So what is the difference between these requests?

This is the working request:

Frame 13: 117 bytes on wire (936 bits), 117 bytes captured (936 bits)
Ethernet II, Src: 2e:4f:24:51:86:41 (2e:4f:24:51:86:41), Dst: Ubiquiti_7a:e6:d2 (24:5a:4c:7a:e6:d2)
Internet Protocol Version 4, Src: 185.233.174.4, Dst: 185.233.175.180
Transmission Control Protocol, Src Port: 58212, Dst Port: 80, Seq: 190, Ack: 1, Len: 51
[2 Reassembled TCP Segments (240 bytes): #12(189), #13(51)]
Hypertext Transfer Protocol
    POST /x?a=add_client HTTP/1.1\r\n
        [Expert Info (Chat/Sequence): POST /x?a=add_client HTTP/1.1\r\n]
            [POST /x?a=add_client HTTP/1.1\r\n]
            [Severity level: Chat]
            [Group: Sequence]
        Request Method: POST
        Request URI: /x?a=add_client
            Request URI Path: /x
            Request URI Query: a=add_client
                Request URI Query Parameter: a=add_client
        Request Version: HTTP/1.1
    Host: infscape-test.cyberfusion.nl\r\n
    Accept-Encoding: identity\r\n
    Content-Length: 51\r\n
        [Content length: 51]
    Accept: application/json\r\n
    Content-Type: application/json; charset=UTF-8\r\n
    \r\n
    [Full request URI: http://infscape-test.cyberfusion.nl/x?a=add_client]
    [HTTP request 1/1]
    [Response in frame: 15]
    File Data: 51 bytes
JavaScript Object Notation: application/json
Line-based text data: application/json (1 lines)
    clientname=test2&ses=x

… and the response:

Frame 15: 71 bytes on wire (568 bits), 71 bytes captured (568 bits)
Ethernet II, Src: Ubiquiti_7a:e6:d2 (24:5a:4c:7a:e6:d2), Dst: 2e:4f:24:51:86:41 (2e:4f:24:51:86:41)
Internet Protocol Version 4, Src: 185.233.175.180, Dst: 185.233.174.4
Transmission Control Protocol, Src Port: 80, Dst Port: 58212, Seq: 550, Ack: 241, Len: 5
[2 Reassembled TCP Segments (554 bytes): #14(549), #15(5)]
Hypertext Transfer Protocol
    HTTP/1.1 200 OK\r\n
        [Expert Info (Chat/Sequence): HTTP/1.1 200 OK\r\n]
            [HTTP/1.1 200 OK\r\n]
            [Severity level: Chat]
            [Group: Sequence]
        Response Version: HTTP/1.1
        Status Code: 200
        [Status Code Description: OK]
        Response Phrase: OK
    Date: Mon, 31 Jul 2023 13:59:57 GMT\r\n
    Server: Apache/2.4.38 (Debian)\r\n
    X-Frame-Options: DENY\r\n
    Vary: Accept-Encoding\r\n
    Cache-Control: no-cache\r\n
     [truncated]Content-Security-Policy: default-src 'none'; script-src 'self'; connect-src 'self'; img-src 'self'; style-src 'self' 'unsafe-hashes' 'sha256-qSZA6CI9aOxHMjlPshgo4JdeLOMbN8Q0mCvhJEW5eLc='; font-src 'self'; block-all-mixed-conte
    Transfer-Encoding: chunked\r\n
    Content-Type: application/json\r\n
    \r\n
    [HTTP response 1/1]
    [Time since request: 0.000722000 seconds]
    [Request in frame: 13]
    [Request URI: http://infscape-test.cyberfusion.nl/x?a=add_client]
    HTTP chunked response
    File Data: 39 bytes
JavaScript Object Notation: application/json
    Object
        Member: already_exists
            [Path with value: /already_exists:true]
            [Member with value: already_exists:true]
            True value
            Key: already_exists
            [Path: /already_exists]
        Member: ok
            [Path with value: /ok:true]
            [Member with value: ok:true]
            True value
            Key: ok
            [Path: /ok]

This is the not working request:

Frame 2: 117 bytes on wire (936 bits), 117 bytes captured (936 bits)
Ethernet II, Src: 2e:4f:24:51:86:41 (2e:4f:24:51:86:41), Dst: Ubiquiti_7a:e6:d2 (24:5a:4c:7a:e6:d2)
Internet Protocol Version 4, Src: 185.233.174.4, Dst: 185.233.175.180
Transmission Control Protocol, Src Port: 33038, Dst Port: 80, Seq: 255, Ack: 1, Len: 51
[2 Reassembled TCP Segments (305 bytes): #1(254), #2(51)]
Hypertext Transfer Protocol
    POST /x?a=add_client HTTP/1.1\r\n
        [Expert Info (Chat/Sequence): POST /x?a=add_client HTTP/1.1\r\n]
            [POST /x?a=add_client HTTP/1.1\r\n]
            [Severity level: Chat]
            [Group: Sequence]
        Request Method: POST
        Request URI: /x?a=add_client
            Request URI Path: /x
            Request URI Query: a=add_client
                Request URI Query Parameter: a=add_client
        Request Version: HTTP/1.1
    Host: infscape-test.cyberfusion.nl\r\n
    User-Agent: python-requests/2.25.1\r\n
    Accept-Encoding: gzip, deflate\r\n
    Accept: application/json\r\n
    Connection: keep-alive\r\n
    Content-Type: application/json; charset=UTF-8\r\n
    Content-Length: 51\r\n
        [Content length: 51]
    \r\n
    [Full request URI: http://infscape-test.cyberfusion.nl/x?a=add_client]
    [HTTP request 1/1]
    [Response in frame: 4]
    File Data: 51 bytes
JavaScript Object Notation: application/json
Line-based text data: application/json (1 lines)
    clientname=test2&ses=x

… and the response:

Frame 4: 86 bytes on wire (688 bits), 86 bytes captured (688 bits)
Ethernet II, Src: Ubiquiti_7a:e6:d2 (24:5a:4c:7a:e6:d2), Dst: 2e:4f:24:51:86:41 (2e:4f:24:51:86:41)
Internet Protocol Version 4, Src: 185.233.175.180, Dst: 185.233.174.4
Transmission Control Protocol, Src Port: 80, Dst Port: 33038, Seq: 622, Ack: 306, Len: 20
[2 Reassembled TCP Segments (641 bytes): #3(621), #4(20)]
Hypertext Transfer Protocol
    HTTP/1.1 200 OK\r\n
        [Expert Info (Chat/Sequence): HTTP/1.1 200 OK\r\n]
            [HTTP/1.1 200 OK\r\n]
            [Severity level: Chat]
            [Group: Sequence]
        Response Version: HTTP/1.1
        Status Code: 200
        [Status Code Description: OK]
        Response Phrase: OK
    Date: Mon, 31 Jul 2023 14:26:13 GMT\r\n
    Server: Apache/2.4.38 (Debian)\r\n
    X-Frame-Options: DENY\r\n
    Vary: Accept-Encoding\r\n
    Content-Encoding: gzip\r\n
    Cache-Control: no-cache\r\n
     [truncated]Content-Security-Policy: default-src 'none'; script-src 'self'; connect-src 'self'; img-src 'self'; style-src 'self' 'unsafe-hashes' 'sha256-qSZA6CI9aOxHMjlPshgo4JdeLOMbN8Q0mCvhJEW5eLc='; font-src 'self'; block-all-mixed-conte
    Keep-Alive: timeout=5, max=100\r\n
    Connection: Keep-Alive\r\n
    Transfer-Encoding: chunked\r\n
    Content-Type: application/json\r\n
    \r\n
    [HTTP response 1/1]
    [Time since request: 0.001211000 seconds]
    [Request in frame: 2]
    [Request URI: http://infscape-test.cyberfusion.nl/x?a=add_client]
    HTTP chunked response
    Content-encoded entity body (gzip): 41 bytes -> 15 bytes
    File Data: 15 bytes
JavaScript Object Notation: application/json
    Object
        Member: error
            [Path with value: /error:1]
            [Member with value: error:1]
            Number value: 1
            Key: error
            [Path: /error]

Differences:

  • The not working request has the User-Agent header. Omitting it does not fix the issue.
  • The not working request uses HTTP keepalive. Disabling it does not fix the issue.
  • The Accept-Encoding header differs. Setting it to the same value does not fix the issue.

Aside from this, another interesting thing occurred to me.

On machine B, the Infscape web interface is used. As the Infscape web interface does API calls, I checked its requests. According to Chrome Dev Tools, this is the cURL equivalent of the request that it does (‘Network’ → right-click request → ‘Copy’ → ‘Copy as cURL’):

curl 'https://infscape-test.cyberfusion.nl/x?a=add_client' \
  -H 'Accept: application/json, text/javascript, */*; q=0.01' \
  -H 'Accept-Language: nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7' \
  -H 'Cache-Control: no-cache' \
  -H 'Connection: keep-alive' \
  -H 'Content-Type: application/json; charset=UTF-8' \
  -H 'Origin: https://infscape-test.cyberfusion.nl' \
  -H 'Pragma: no-cache' \
  -H 'Referer: https://infscape-test.cyberfusion.nl/index.htm?_=16908141477284631' \
  -H 'Sec-Fetch-Dest: empty' \
  -H 'Sec-Fetch-Mode: cors' \
  -H 'Sec-Fetch-Site: same-origin' \
  -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36' \
  -H 'X-Requested-With: XMLHttpRequest' \
  -H 'sec-ch-ua: "Not/A)Brand";v="99", "Google Chrome";v="115", "Chromium";v="115"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "macOS"' \
  --data-raw 'clientname=test3&ses=x&lang=en' \
  --compressed

On machine A, this does not work:

*   Trying 185.233.175.180:443...
* Connected to infscape-test.cyberfusion.nl (185.233.175.180) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=infscape-test.cyberfusion.nl
*  start date: Jul 31 11:44:32 2023 GMT
*  expire date: Oct 29 11:44:31 2023 GMT
*  subjectAltName: host "infscape-test.cyberfusion.nl" matched cert's "infscape-test.cyberfusion.nl"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
> POST /x?a=add_client HTTP/1.1
> Host: infscape-test.cyberfusion.nl
> Accept-Encoding: deflate, gzip, br
> Accept: application/json, text/javascript, */*; q=0.01
> Accept-Language: nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
> Cache-Control: no-cache
> Connection: keep-alive
> Content-Type: application/json; charset=UTF-8
> Origin: https://infscape-test.cyberfusion.nl
> Pragma: no-cache
> Referer: https://infscape-test.cyberfusion.nl/index.htm?_=16908141477284631
> Sec-Fetch-Dest: empty
> Sec-Fetch-Mode: cors
> Sec-Fetch-Site: same-origin
> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36
> X-Requested-With: XMLHttpRequest
> sec-ch-ua: "Not/A)Brand";v="99", "Google Chrome";v="115", "Chromium";v="115"
> sec-ch-ua-mobile: ?0
> sec-ch-ua-platform: "macOS"
> Content-Length: 59
>
* upload completely sent off: 59 out of 59 bytes
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Mon, 31 Jul 2023 15:04:09 GMT
< Server: Apache/2.4.38 (Debian)
< X-Frame-Options: DENY
< Vary: Accept-Encoding
< Content-Encoding: gzip
< Cache-Control: no-cache
< Content-Security-Policy: default-src 'none'; script-src 'self'; connect-src 'self'; img-src 'self'; style-src 'self' 'unsafe-hashes' 'sha256-qSZA6CI9aOxHMjlPshgo4JdeLOMbN8Q0mCvhJEW5eLc='; font-src 'self'; block-all-mixed-content; form-action 'self'; base-uri 'self'; frame-src 'self'
< Strict-Transport-Security: max-age=31536000
< Keep-Alive: timeout=5, max=100
< Connection: Keep-Alive
< Transfer-Encoding: chunked
< Content-Type: application/json
<
{
"error": 1
}
* Connection #0 to host infscape-test.cyberfusion.nl left intact

But on machine B, this does work:

*   Trying 185.233.175.180:443...
* Connected to infscape-test.cyberfusion.nl (185.233.175.180) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (OUT), TLS handshake, Client hello (1):
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=infscape-test.cyberfusion.nl
*  start date: Jul 31 11:44:32 2023 GMT
*  expire date: Oct 29 11:44:31 2023 GMT
*  subjectAltName: host "infscape-test.cyberfusion.nl" matched cert's "infscape-test.cyberfusion.nl"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
> POST /x?a=add_client HTTP/1.1
> Host: infscape-test.cyberfusion.nl
> Accept-Encoding: deflate, gzip
> Accept: application/json, text/javascript, */*; q=0.01
> Accept-Language: nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
> Cache-Control: no-cache
> Connection: keep-alive
> Content-Type: application/json; charset=UTF-8
> Origin: https://infscape-test.cyberfusion.nl
> Pragma: no-cache
> Referer: https://infscape-test.cyberfusion.nl/index.htm?_=16908141477284631
> Sec-Fetch-Dest: empty
> Sec-Fetch-Mode: cors
> Sec-Fetch-Site: same-origin
> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36
> X-Requested-With: XMLHttpRequest
> sec-ch-ua: "Not/A)Brand";v="99", "Google Chrome";v="115", "Chromium";v="115"
> sec-ch-ua-mobile: ?0
> sec-ch-ua-platform: "macOS"
> Content-Length: 59
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Mon, 31 Jul 2023 15:04:15 GMT
< Server: Apache/2.4.38 (Debian)
< X-Frame-Options: DENY
< Vary: Accept-Encoding
< Content-Encoding: gzip
< Cache-Control: no-cache
< Content-Security-Policy: default-src 'none'; script-src 'self'; connect-src 'self'; img-src 'self'; style-src 'self' 'unsafe-hashes' 'sha256-qSZA6CI9aOxHMjlPshgo4JdeLOMbN8Q0mCvhJEW5eLc='; font-src 'self'; block-all-mixed-content; form-action 'self'; base-uri 'self'; frame-src 'self'
< Strict-Transport-Security: max-age=31536000
< Keep-Alive: timeout=5, max=100
< Connection: Keep-Alive
< Transfer-Encoding: chunked
< Content-Type: application/json
<
{
"already_exists": true,
"ok": true
}
* Connection #0 to host infscape-test.cyberfusion.nl left intact

But there is no substantiative difference between both requests.

2

The session is locked to the user agent and IP address it was created with. See:

3

More helpful API error messages would be, well, helpful :slight_smile:

Thank you @uroni.

4

The session is locked to the user agent and IP address it was created with.

How do I disable IP locking?