Archive for the ‘Networking’ Category

Everyday Ubuntu commands

February 4, 2020

Host IP:

hostname -I

Disks:

df

Tasks:

top

Processes:

ps -ef | grep whatever

Find the largest files and dirs:

sudo du -a / | sort -n -r | head -n 20

to be continued…

Troubleshooting network connection

May 24, 2018

How to troubleshoot network connection between AWS Lambda, AWS EC2 through Application Load Balancer and outside world.

First try

curl https://thehost.com

returns empty.

Second try

curl --dump-header - https://thehost.com

returns something more meaningful but a bit misleading:

HTTP/1.1 302 Found
 Date: Thu, 24 May 2018 01:15:46 GMT
 Content-Type: text/plain; charset=utf-8
 Content-Length: 0
 Connection: keep-alive
 Location: login_page

telnet connects though:

telnet thehost 443
Trying 55.66.222.111...
Connected to thehost.com.
Escape character is '^]'.
^]
telnet> status
Connected to thehost.com.
Operating in obsolete linemode
Local character echo
Escape character is '^]'.
Connection closed by foreign host

Third try

curl -svo /dev/null https://thehost.com

gives us a hint with TLS info and the root of a problem below:

* Rebuilt URL to: https://thehost.com/
 * Trying 54.66.241.172...
 * Connected to thehost.com (55.66.222.111) port 443 (#0)
 * found 148 certificates in /etc/ssl/certs/ca-certificates.crt
 * found 592 certificates in /etc/ssl/certs
 * ALPN, offering http/1.1
 * SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
 * server certificate verification OK
 * server certificate status verification SKIPPED
 * common name: *.thehost.com (matched)
 * server certificate expiration date OK
 * server certificate activation date OK
 * certificate public key: RSA
 * certificate version: #3
 * subject: CN=*.thehost.com
 * start date: Thu, 22 Mar 2018 00:00:00 GMT
 * expire date: Mon, 22 Apr 2019 12:00:00 GMT
 * issuer: C=US,O=Amazon,OU=Server CA 1B,CN=Amazon
 * compression: NULL
 * ALPN, server accepted to use http/1.1
 > GET / HTTP/1.1
 > Host: thehost.com
 > User-Agent: curl/7.47.0
 > Accept: */*
 >
 < HTTP/1.1 502 Bad Gateway
 < Server: awselb/2.0
 < Date: Thu, 24 May 2018 01:25:00 GMT
 < Content-Type: text/html
 < Content-Length: 138
 < Connection: keep-alive

Referring to the doco, and here is the answer:

ALB does support HTTP/2, but only for HTTPS listeners. You will not be able to send plaintext HTTP/2 requests.

So, no traffic will go through HTTP/2 AWS ALB unless it’s on 443 port. Damn!

References:

  1. https://forums.aws.amazon.com/thread.jspa?threadID=122901
  2. https://forums.aws.amazon.com/thread.jspa?threadID=238804
  3. https://forums.aws.amazon.com/thread.jspa?threadID=263770
  4. https://forums.aws.amazon.com/thread.jspa?threadID=94483

P.S.

Tricks to try next time:

curl -L http://example.com/
wget -S -O /dev/null http://www.example.com
wget -s -O /dev/null --header="Host: foo.bar" http://www.example.com
openssl s_client -connect thehost.com:443
curl -v --http2 https://www.example.com

Another possible reason:

There were no “Content-Length” and “Transfer-Encoding” headers in the response, and the backend used keep-alive and didn’t close connection as ELB was expecting it to do. Placing Apache in between backend and ELB should solve this problem (I haven’t tested it yet).