All posts by Daniel Stenberg

cURL and libcurl

curl ootw: –mail-from

(older options of the week)

--mail-from has no short version. This option was added to curl 7.20.0 in February 2010.

I know this surprises many curl users: yes curl can both send and receive emails.

SMTP

curl can do “uploads” to an SMTP server, which usually has the effect that an email is delivered to somewhere onward from that server. Ie, curl can send an email.

When communicating with an SMTP server to send an email, curl needs to provide a certain data set and one of the mandatory fields of data that is necessary to provide is the email of the sender.

It should be noted that this is not necessary the same email as the one that is displayed in the From: field that the recipient’s email client will show. But it can be the same.

Example

curl --mail-from from@example.com --mail-rcpt to@example.com smtp://mail.example.com -T body.txt

Receiving emails

You can use curl to receive emails over POP3 or IMAP. SMTP is only for delivering email.

Related options

--mail-rcpt for the recipient(s), and --mail-auth for the authentication address.

cURL and libcurl

curl is 8000 days old

Another pointless number that happens to be round and look nice so I feel a need to highlight it.

When curl was born WiFi didn’t exist yet. Smartphones and tablets weren’t invented. Other things that didn’t exist include YouTube, Facebook, Twitter, Instagram, Firefox, Chrome, Spotify, Google search, Wikipedia, Windows 98 or emojis.

curl was born in a different time, but also in the beginning of the explosion of the web and Internet Protocols. Just before the big growth wave.

In 1996 when I started working on the precursor to curl, there were around 250,000 web sites (sources vary slightly)..

In 1998 when curl shipped, the number of sites were already around 2,400,000. Ten times larger amount in just those two years.

In early 2020, the amount of web sites are around 1,700,000,000 to 2,000,000,000 (depending on who provides the stats). The number of web sites has thus grown at least 70,000% over curl’s 8000 days of life and perhaps as much as 8000 times the amount as when I first working with HTTP clients.

One of the oldest still available snapshots of the curl web site is from the end of 1998, when curl was just a little over 6 months old. On that page we can read the following:

That “massive popularity” looks charming and possibly a bit naive today. The number of monthly curl downloads have also possibly grown by 8,000 times or so – by estimates only, as most users download curl from other places than our web site these days. Even more users get it installed as part of their OS or bundled with something else.

Thank you for flying curl.

(This day occurs only a little over a month before curl turns 22, there will be much more navel-gazing then, I promise.)

Image by Annie Spratt from Pixabay

cURL and libcurl

curl ootw: –keepalive-time

(previously blogged about options are listed here.)

This option is named --keepalive-time even if the title above ruins the double-dash (thanks for that WordPress!). This command line option was introduced in curl 7.18.0 back in early 2008. There’s no short version of it.

The option takes a numerical argument; number of seconds.

What’s implied in the option name and not spelled out is that the particular thing you ask to keep alive is a TCP connection. When the keepalive feature is not used, TCP connections typically don’t send anything at all if no data is transmitted.

Idle TCP connections

Silent TCP connections typically cause the two primary issues:

  1. Middle-boxes that track connections, such as your typical NAT boxes (home WiFi routers most notoriously) will consider silent connections “dead” after a certain period of time and drop all knowledge about them, leading to the connection non functioning when the client (or server) later wants to resume operation of it.
  2. Neither side of the connection will notice when the network between them breaks, as it takes actual traffic to do so. This is of course also a feature, because there’s no need to be alarmed by a breakage if there’s no traffic as it might be fine again when it eventually gets used again.

TCP stacks then typically implement a low-level feature where they can send a “ping” frame over the connection if it has been idle for a certain amount of time. This is the keepalive packet.

--keepalive-time <seconds> therefor sets the interval. After this many seconds of “silence” on the connection, there will be a keepalive packet sent. The packet is totally invisible to the applications on both sides but will maintain the connection through NATs better and if the connection is broken, this packet will make curl detect it.

Keepalive is not always enough

To complicate issues even further, there are also devices out there that will still close down connections if they only send TCP keepalive packets and no data for certain period. Several protocols on top of TCP have their own keepalive alternatives (sometimes called ping) for this and other reasons.

This aggressive style of closing connections without actual traffic TCP traffic typically hurts long-going FTP transfers. This, because FTP sets up two connections for a transfer, but the first one is the “control connection” and while a transfer is being delivered on the “data connection”, nothing happens over the first one. This can then result in the control connection being “dead” by the time the data transfer completes!

Default

The default keepalive time is 60 seconds. You can also disable keepalive completely with the --no-keepalive option.

The default time has been selected to be fairly low because many NAT routers out there in the wild are fairly aggressively and close idle connections already after two minutes (120) seconds.

For what protocols

This works for all TCP-based protocols, which is what most protocols curl speaks use. The only exception right now is TFTP. (See also QUIC below.)

Example

Change the interval to 3 minutes:

curl --keepalive-time 180 https://example.com/

Related options

A related functionality is the --speed-limit amd --speed-time options that will cancel a transfer if the transfer speed drops below a given speed for a certain time. Or just the --max-time that sets a global timeout for an entire operation.

QUIC?

Soon we will see QUIC getting used instead of TCP for some protocols: HTTP/3 being the first in line for that. We will have to see what exactly we do with this option when QUIC starts to get used and what the proper mapping and behavior shall be.

Rockbox

Rockbox services transition

1 Comment

Remember Rockbox? It is a free software firmware replacement for mp3 players. I co-founded the project back in 2001 together with Björn and Linus. I officially left the project back in 2014.

The project is still alive today, even of course many of us can’t hardly remember the concept of a separate portable music player and can’t figure out why that’s a good idea when we carry around a powerful phone all days anyway that can do the job – better.

Already when the project took off, we at Haxx hosted the web site and related services. Heck, if you don’t run your own server to add fun toy projects to, then what kind of lame hacker are you?

None of us in Haxx no longer participates in the project and we haven’t done so for several years. We host the web site, we run the mailing lists, we take care of the DNS, etc.

Most of the time it’s no biggie. The server hosts a bunch of other things anyway for other project so what is a few extra services after all?

Then there are times when things stop working or when we get a refreshed bot attack or web crawler abuse against the site and we get reminded that here we are more than eighteen years later hosting things and doing work for a project we don’t care much for anymore.

It doesn’t seem right anymore. We’re pulling the plug on all services for Rockbox that occasionally gives us work and annoyances. We’re offering to keep hosting DNS and the mailing lists – but if active project members rather do those too, feel free. It never was a life-time offer and the time has come for us.

If people still care for the project, it is much better if those people will also care for these things for the project’s sake. And today there are more options than ever for an open source project to get hosting, bug tracking, CI systems etc setup for free with quality. There’s no need for us ex-Rockboxers to keep doing this job that we don’t want to do.

I created a wiki page to detail The Transition. We will close down the specified services on January 1st 2021 but I strongly urge existing Rockboxers to get the transition going as soon as possible.

I’ve also announced this on the rockbox-dev mailing list, and I’ve mentioned it in the Rockbox IRC.

cURL and libcurl

curl ootw: -U for proxy credentials

(older options of the week)

-U, --proxy-user

The short version of this option uses the uppercase letter ‘U’. It is important since the lower case letter ‘u’ is used for another option. The longer form of the option is spelled --proxy-user.

This command option existed already in the first ever curl release!

The man page‘s first paragraph describes this option as:

Specify the user name and password to use for proxy authentication.

Proxy

This option is for using a proxy. So let’s first briefly look at what a proxy is.

A proxy is a “middle man” in the communication between a client (curl) and a server (the one that holds the contents you want to download or will receive the content you want to upload).

The client communicates via this proxy to reach the server. When a proxy is used, the server communicates only with the proxy and the client also only communicates with the proxy:

curl <===> proxy <===> server

There exists several different types of proxies and a proxy can require authentication for it to allow it to be used.

Proxy authentication

Sometimes the proxy you want or need to use requires authentication, meaning that you need to provide your credentials to the proxy in order to be allowed to use it. The -U option is used to set the name and password used when authenticating with the proxy (separated by a colon).

You need to know this name and password, curl can’t figure them out – unless you’re on Windows and your curl is built with SSPI support as then it can magically use the current user’s credentials if you provide blank credentials in the option: -U :.

Security

Providing passwords in command lines is a bit icky. If you write it in a script, someone else might see the script and figure them out.

If the proxy communication is done in clear text (for example over HTTP) some authentication methods (for example Basic) will transmit the credentials in clear text across the network to the proxy, possibly readable by others.

Command line options may also appear in process listing so other users on the system can see them there – although curl will attempt to blank them out from ps outputs if the system supports it (Linux does).

Needs other options too

A typical command line that use -U also sets at least which proxy to use, with the -x option with a URL that specifies which type of proxy, the proxy host name and which port number the proxy runs on.

If the proxy is a HTTP or HTTPS type, you might also need to specify which type of authentication you want to use. For example with --proxy-anyauth to let curl figure it out by itself.

If you know what HTTP auth method the proxy uses, you can also explicitly enable that directly on the command line with the correct option. Like for example --proxy-basic or --proxy-digest.

SOCKS proxies

curl also supports SOCKS proxies, which is a different type than HTTP or HTTPS proxies. When you use a SOCKS proxy, you need to tell curl that, either with the correct prefix in the -x argument or by with one of the --socks* options.

Example command line

curl -x http://proxy.example.com:8080 -U user:password https://example.com

See Also

The corresponding option for sending credentials to a server instead of proxy uses the lowercase version: -u / --user.

cURL and libcurl, Security

Remote-exploiting curl

In a Blackhat 2019 presentation, three gentlemen from the Tencent Blade Team explained how they found and managed to exploit two curl flaws. Both related to NTLM over HTTP. The “client version Heartbleed” as they call it.

Reported responsibly

The Tencent team already reported the bugs responsibly to us and we already fixed them back in February 2019, but the talk is still very interesting I think.

From my point of view, as I have already discussed these bugs with the team when they were reported us and when I worked on fixing them, I find it very interesting and educational to learn more about how exactly they envision an attacker would go about and exploit them in practice. I have much too bad imagination sometimes to really think of how bad exactly the problems can end up when a creative attacker gets to play with them.

The security issues

The two specific issues these stellar gents found are already fixed since curl 7.64.0 and you can read all the gory details about them here: CVE-2018-16890 and CVE-2019-3822. The latter is clearly the worse issue.

For all I know, these exploits have never been seen or reported to happen in real life.

Upgrade?

Luckily, most distros that ship older curl versions still back-port and apply later security patches so even if you may see that you have an older curl version installed on your system, chances are it has already been patched. Of course there’s also a risk that it hasn’t, so you should probably make sure rather than presume…

The video

The slides from their presentation. (The talk also details SQLite issues but they’re completely separate from the curl ones.)

Bug Bounty?

Unfortunately, I’m sorry to admit that these excellent friends of ours did not get a bug bounty from us! 🙁

We got their reports before our bug bounty was setup and we didn’t have neither the means nor the methods to reward them back then. If someone would report such serious bugs now, only a year later, we would probably reward new such findings with several thousand dollars.

On NTLM

NTLM was always wrong, bad and a hack. It’s not an excuse for having bugs in our code but man if someone could just please make that thing go away…

Network, Web

HTTP/3 for everyone

FOSDEM 2020 is over for this time and I had an awesome time in Brussels once again.

Stickers

I brought a huge collection of stickers this year and I kept going back to the wolfSSL stand to refill the stash and it kept being emptied almost as fast. Hundreds of curl stickers were given away! The photo on the right shows my “sticker bag” as it looked before I left Sweden.

Lesson for next year: bring a larger amount of stickers! If you missed out on curl stickers, get in touch and I’ll do my best to satisfy your needs.

The talk

“HTTP/3 for everyone” was my single talk this FOSDEM. Just two days before the talk, I landed updated commits in curl’s git master branch for doing HTTP/3 up-to-date with the latest draft (-25). Very timely and I got to update the slide mentioning this.

As I talked HTTP/3 already last year in the Mozilla devroom, I also made sure to go through the slides I used then to compare and make sure I wouldn’t do too much of the same talk. But lots of things have changed and most of the content is updated and different this time around. Last year, literally hundreds of people were lining up outside wanting to get into room when the doors were closed. This year, I talked in the room Janson, which features 1415 seats. The biggest one on campus. It was pack full!

It is kind of an adrenaline rush to stand in front of such a wall of people. At one time in my talk I paused for a brief moment and then I felt I could almost hear the complete silence when a huge amount of attentive faces captured what I had to say.

The audience, photographed by Sidsel Jensen who had to sit in the stairs…
Photo by Mirza Krak
Photo by Wolfgang Gassler

I got a lot of positive feedback on the presentation. I also thought that my decision to not even try to take question in the big room was a correct and I ended up talking and discussing details behind the scene for a good while after my talk was done. Really fun!

The video

The video is also available from the FOSDEM site in webm and mp4 formats.

The slides

If you want the slides only, run over to slideshare and view them.

cURL and libcurl

curl ootw: -k really means insecure

4 Comments

(ootw stands for option of the week)

Long form: --insecure. The title is a little misleading, but this refers to the lowercase -k option.

This option has existed in the curl tool since the early days, and has been frequently misused ever since. Or I should perhaps call it “overused”.

Truly makes a transfer insecure

The name of the long form of this option was selected and chosen carefully to properly signal what it does: it makes a transfer insecure that could otherwise be done securely.

The option tells curl to skip the verification of the server’s TLS certificate – it will skip the cryptographic certificate verification (that it was signed by a trusted CA) and it will skip other certificate checks, like that it was made for the host name curl connects to and that it hasn’t expired etc.

For SCP and SFTP transfers, the option makes curlskip the known hosts verification. SCP and SFTP are SSH-based and they don’t use the general CA based PKI setup.

If the transfer isn’t using TLS or SSH, then this option has no effect or purpose as such transfers are already insecure by default.

Unable to detect MITM attacks

When this option is used, curl cannot detect man-in-the-middle attacks as it longer checks that it actually connects to the correct server. That is insecure.

Slips into production

One of the primary reasons why you should avoid this option in your scripts as far as possible is that it is very easy to let this slip through and get shipped into production scripts and then you’ve practically converted perfectly secure transfers into very insecure ones.

Instead, you should work on getting an updated CA cert bundle that holds certificates so that you can verify your server. For test and local servers you can get the server cert and use that to verify it subsequently.

Optionally, if you run your production server on a test or local server and you just have a server name mismatch, you can fix that in your test scripts too just by telling curl what server to use.

libcurl options

In libcurl speak, the -k functionality is accomplished with two different options: CURLOPT_SSL_VERIFYHOST and CURLOPT_SSL_VERIFYPEER. You should not allow your application to set them to 0.

Since many bindings to libcurl use the same option names you can also find PHP programs etc setting these to zero, and you should always treat that as a warning sign!

Why does it exist?

Every now and then people suggest we should remove this option.

It does serve a purpose in the chicken and egg scenario where you don’t have a proper certificate locally to verify your server against and sometimes the server you want to quickly poll is wrongly configured and a proper check fails.

How do you use it?

curl -k https://example.com

You’d often use it in combination with -v to view the TLS and certificate information in the handshake so that you can fix it and remove -k again.

Related options

--cacert tells curl where to find the CA cert bundle. If you use a HTTPS proxy and want the -k functionality for the proxy itself you want --proxy-insecure.

An alternative approach to using a CA cert bundle for TLS based transfers, is to use public key pinning, with the --pinnedpubkey option.

Open Source, wolfSSL

Coming to FOSDEM 2020

I’m going to FOSDEM again in 2020, this will be my 11th consecutive year I’m travling to this awesome conference in Brussels, Belgium.

At this my 11th FOSDEM visit I will also deliver my 11th FOSDEM talk: “HTTP/3 for everyone“. It will happen at 16:00 Saturday the 1st of February 2020, in Janson, the largest room on the campus. (My third talk in the main track.)

For those who have seen me talk about HTTP/3 before, this talk will certainly have overlaps but I’m also always refreshing and improving slides and I update them as the process moves on, things changes and I get feedback. I spoke about HTTP/3 already at FODEM 2019 in the Mozilla devroom (at which time there was a looong line of people who tried, but couldn’t get a seat in the room) – but I think you’ll find that there’s enough changes and improvements in this talk to keep you entertained this year as well!

If you come to FOSDEM, don’t hesitate to come say hi and grab a curl sticker or two – I intend to bring and distribute plenty – and talk curl, HTTP and Internet transfers with me!

You will most likely find me at my talk, in the cafeteria area or at the wolfSSL stall. (DM me on twitter to pin me down! @bagder)

cURL and libcurl

curl cheat sheet refresh

1 Comment

Several years ago I made a first version of a “curl HTTP cheat sheet” to facilitate the most common curl command line options when working with HTTP.

This has now been refreshed after I took lots of feedback from friends on twitter, and then worked on rearranging the lines and columns so that it got as compact as possible without sacrificing readability (too much).

See the updated online version here.

The curl HTTP cheat sheet, 2020 edition.

The plan is to make stickers out of this – and possibly t-shirts as well. I did some test prints and deemed that with a 125 mm width, all the text is still clearly readable.

If things go well, I’ll hand out these beauties at curl up 2020 and of course leftovers will then be given away to whoever I run into at other places and conferences where I bring stickers…