SMTP is a line–based protocol, including the part that transfers the message body
The server needs to parse the message headers, so it can't be an opaque blob. If the client uses IMAP, the server needs to fully parse the message. The only alternative is POP3, where the client downloads all messages as blobs and you can only read your email from one location, which made sense in the year 2000 but not now when everyone has several devices.
POP3 is more for reading and acting on your email in one place (taking notes, plan actions, discard and delete,…). No need to consume them on other devices as you’ve already extracted the important bits.
I use imap on my mobile device, but that’s mostly for recent emails until I get to my computer. Then it’s downloaded and deleted from the server.
POP is a simple mail transfer protocol (hehe...). It supports three things: get number of mails, download mail by number, delete mail by number. This is what you need to move mails in bulk from one point to another. POP3 mail clients are local maildir clients that use POP3 to get new mail from the server. It's like SMTP if it were based on polling.
IMAP is an interactive protocol that is closer to the interaction between Gmail frontend and backend. It does many things. The client implements a local view of a central source of truth.
No, the difference is that IMAP doesn't store anything other than headers on the client (at least, not until the user tries to read a message), while POP3 eagerly downloads messages whenever they're available. A POP3 client can be configured with various remote retention policies, or even to never delete downloaded messages.
I don't have an IMAP account available to check, but AFAIK, you should not have locally the content of any message you've never read before. The whole point of IMAP is that it doesn't download messages, but instead acts like a window into the server.
Not at all. IMAP can do a lot of complex operations on the email while leaving it on the server, for example you can have the server search the email, flag it (mark it important, or read, or unread).
The idea with IMAP is multiple clients can work with your email - for example your desktop and your phone can both see the same messages and manipulate them, even offline.
Gmail basically is IMAP with a couple extras, and your desktop (via a browser) and your phone (via a dedicated app) can both see the same messages. Only the phone can work offline though, because there is little demand for a dedicated desktop email client, it's always via a browser. But Google could easily make such a thing if they wanted.
But everything after headers can (almost) be a blob. Just copy buffers while taking care to track CRLF and look if what follows is a space. In fact, you have to do it anyhow, because line-folding is allowed in headers as well! And this "chunking long lines" technique has been around since the 70s, when people started writing parsers on top of hand-crafted buffered I/O.
Mails are (or used to be) processed line-by-line, typically using fixed-length buffers. This avoids dynamic memory allocation and having to write a streaming parser. RFC 821 finally limited the line length to at most 1000 bytes.
Given a mechanism for soft line breaks, breaking already at below 80 characters would increase compatibility with older mail software and be more convenient when listing the raw email in a terminal.
This is also why MIME Base64 typically inserts line breaks after 76 characters.
In early days, many/most people also read their email on terminals (or printers) with 80-column lines, so breaking lines at 72-ish was considered good email etiquette (to allow for later quoting prefix ">" without exceeding 80 characters).
I don't think kids today realize how little memory we had when SMTP was designed.
For example, the PDP-11 (early 1970s), which was shared among dozens of concurrent users, had 512 kilobytes of RAM. The VAX-11 (late 1970s) might have as much as 2 megabytes.
Programmers were literally counting bytes to write programs.
My point is that bytes mattered. If you could put a year in 2 bytes instead of 4, you did. If you could shrink the TCP header by packing fields, you did. And if you could limit SMTP memory use by specifying a 1000-byte limit, then that's what you did.
Every programmer I know from that era knew how big things were in bytes, because it mattered.
Also, not all PDP-11 systems had VM. And the designers of SMTP certainly did not expect that it would only run on systems with VM.
This is how email work(ed) over smtp. When each command was sent it would get a '200'-class message (success) or 400/500-class message (failure). Sound familiar?
Yep! And also, if you included a blank line and then the headers for a new email in the bottom of your message, you could tell the server, hey, here comes another email for you to process!
If you were typing into a feedback form powered by something from Matt’s Script Archive, there was about a 95% chance you could trivially get it to send out multiple emails to other parties for every one email sent to the site’s owner.
Back in 80s-90s it was common to use static buffers to simplify implementation - you allocate a fixed size buffer and reject a message if it has a line longer than the buffer size. SMTP RFC specifies 1000 symbols limit (including \r\n) but it's common to wrap around 87 symbols so it is easy to examine source (on a small screen).
"BITNET was a co-operative university computer network in the United States founded in 1981 by Ira Fuchs at the City University of New York (CUNY) and Greydon Freeman at Yale University."
BITNET connected mainframes, had gateways to the Unix world and was still active in the 90s. And limited line lengths … some may remember SYSIN DD DATA … oh my goodness …
The simplest reason: Mail servers have long had features which will send the mail client a substring of the text content without transferring the entire thing. Like the GMail inbox view, before you open any one message.
I suspect this is relevant because Quoted Printable was only a useful encoding for MIME types like text and HTML (the human readable email body), not binary (eg. Attachments, images, videos). Mail servers (if they want) can effectively treat the binary types as an opaque blob, while the text types can be read for more efficient transfer of message listings to the client.
As far as I can remember, most mail servers were fairly sane about that sort of thing, even back in the 90’s when this stuff was introduced. However, there were always these more or less motivated fears about some server somewhere running on some ancient IBM hardware using EBCDIC encoding and truncating everything to 72 characters because its model of the world was based on punched cards. So standards were written to handle all those bizarre systems. And I am sure that there is someone on HN who actually used one of those servers...
RFC822 explicitly says it is for readability on systems with simple display software. Given that the protocol is from 1982 and systems back then had between 4 and 16kb RAM in total it might have made sense to give the lower end thin client systems of the day something preprocessed.
Also it is an easy way to stop a denial of service attack. If you let an infinite amount in that field. I can remotely overflow your system memory. The mail system can just error out and hang up on the person trying the attack instead of crashing out.
Keep in mind that in ye olden days, email was not a worldwide communication method. It was more typical for it to be an internal-only mail system, running on whatever legacy mainframe your org had, and working within whatever constraints that forced. So in the 90s when the internet began to expand, and email to external organizations became a bigger thing, you were just as concerned with compatibility with all those legacy terminal-based mail programs, which led to different choices when engineering the systems.
Are you certain? Not OP, but a huge chunk of early RFCs was about how to let giant IBM systems talk to everyone else, specifying everything from character sets (nearly universally “7-bit ASCII”) to end of line/message characters. Otherwise, IBM would’ve tried to make EBCDIC the default for everything.
For instance, consider FTP’s text mode, which was primarily a way to accidentally corrupt your download when you forgot to type “bin” first, but was also handy for getting human readable files from one incompatible system to another.
My first reading was that you were disagreeing with the bits about email worrying about compatibility, and that part seemed reasonably true to me.
As to the other bits, I think even in the uucp era, email was mostly internal, by volume of mail sent, even though you could clearly talk to remote sites if everything was set up correctly. It was capable of being a worldwide communication system. I bet the local admins responsible for monitoring the telephone bill preferred to keep that in check, though.
It's probably the most complex and impossible to solve take, it's not even true in the US or any other place in the world.
It doesn't even work in a single country for the simple reason that governments have very different ideas how to redistribute taxes. If one country can't do it well, how could the EU?
I remember how react team's message, around the time hooks were introduced, was how hooks were going to save us from the tyranny of `this`, which people presumably found confusing.
I often think back to that message, while adding things in a dependency array. Especially those things that I know won't change (e.g. the redux `dispatch` function pulled from the context), but the linter doesn't. Or while being admonished by the linter for reading from a ref, or writing to it.
We are actively improving on the performance! Also, I am a previous code canvas user too, but I felt like it didn't help me understand my codebase as much as I wanted it to.. that's why I decided to experiment something myself! :)
Thanks, CC is really helpful already for me the more tools like these exists, the better. With AI coding agents, I keep watching the visualization rather than the changeset first.
reply