Concatenated SMS

SMS (Short Message Service) messages, also known as text messages, have a maximum length of 1120 bits. Depending on the content of the message, it can be encoded with various lengths of bits equaling one character.

If 7-bit encoding is used, it allows for 160 characters (1120 / 7) in the SMS, with an alphabet size of 128 letters (2^7). This works well for most English communication. There are also different 7-bit alphabets for other languages that can be specified in the SMS header. Depending on how much information is encoded in the header, enough data is left for between 150-155 characters in the SMS.

If 8-bit encoding is used, it allows for 140 characters (1120 / 8) in the SMS, with an alphabet size of 256 letters (2^8). 8-bit encoding is probably not currently in use by any SMS system.

If 16-bit encoding is used, it allows for 70 characters (1120 / 16) in the SMS, with an alphabet size of 65536 letters (2^16). This is known as as UCS-2 (Universal Character Set-2) encoding. It has largely been superseded by UTF-16.

UTF-16 (Unicode Transformation Format-16) uses a variable 16-32 bit encoding scheme. The most common letters are represented by 16-bit units. Some of the bits are reserved as markers to indicated that the current letter requires 32 bits, which allows it to expand for less common characters. UTF-16 is capable of encoding 1,112,064 letters, including all your favorite Emoji. Most cell phones automatically switch between 7-bit and UTF-16 encoding when sending a SMS depending on the characters being encoded. If you are a teenager, it is possible that every text message you have ever sent had to be encoded in UTF-16 ☺.

All of this is background for the topic of concatenated SMS. Despite what Twitter might believe, not every thought worth expressing can be compressed to 140 characters. Concatenated SMS is a clever scheme where long text messages are split into smaller SMS with a header that indicates how many SMS comprise the message and which one this is. The receiving device reassembles the message from the individual SMS.

At least that is the way it is supposed to work.

I live in the United States and use T-Mobile as my carrier. Recently I was engrossed in a stimulating conversation with friend on Verizon, when he informed me that the last several text messages I had sent him had been received as 63 text messages on his device. Curious, I decided to do a little digging. On the left is a concatenated SMS sent from T-Mobile. On the right is how it is received by Verizon. Note that it makes no difference if the devices are running Android or iOS. If iMessage is turned off so that it forces the use of SMS, even two iPhones will display the same symptoms.

 

However, what makes this more interesting is what happens when a concatenated SMS is sent from Verizon to T-Mobile.

After testing, the four major US carriers, the following behavior was observed.

From the results, the following can be deduced.

  1. AT&T and T-Mobile fully support concatenated SMS on their networks.
  2. Verizon at least partially supports concatenated SMS on messages that originate and terminate within their network. It is unclear if the problems that were reported relate to issues with the phones or the network. The majority of testers did not have issues.
  3. At the SMS hand-off between carriers, Verizon does not correctly handle received concatenated SMS headers. Therefore, when a concatenated SMS is sent from AT&T or T-Mobile to Verizon, it is received as multiple, individual SMS messages.
  4. SMS headers are correctly preserved when sent from Verizon to AT&T or T-Mobile.
  5. Sprint does not handle concatenated SMS at all. Each SMS segment is processed individually. Because the SMS system includes no guarantee that messages are delivered in the order they are sent, often the segments are displayed out of order on the receiving device. This was observed in messages between Sprint and Verizon and would likely be observed if enough test messages were sent between Sprint and the other carriers.

It is not surprising that if the message is transmitted via MMS (Multimedia Messaging Service) instead of concatenated SMS, it comes through correctly on all networks. This is because the content of MMS messages are uploaded to a web server, and then a control SMS is sent to the receiving phone with a link to download the MMS. The receiving phone recognizes the control SMS, downloads the content from the MMS web server, and displays it on the screen. The following message was sent as an MMS because it was a group message with two recipients. Attaching a picture will also force MMS for the transport.

Because concatenated SMS are handled correctly by Verizon within their network and are sent correctly to AT&T and T-Mobile, it probably isn’t intentional that it is mishandled on inbound concatenated SMS from other carriers. Rather, it could easily be a bug in the receiving code that just hasn’t been brought to the attention of the correct engineer. Likewise, Sprint’s lack of support for concatenated SMS could simply be that the correct person has not noticed this feature is lacking. In both cases this is a problem that could likely be fixed relatively easily.

All my friends on Verizon and Sprint would be eternally grateful.

Update

I was recently contacted by a person claiming to have done extensive testing in 2015 showing that concatenated SMS messages sent to Verizon from other networks did indeed come through correctly, but only depending on the devices used. Particularly, he asserted that if both the sending and the receiving devices ran sufficiently new versions of Android concatenated SMS would work. I decided to revisit my testing to see if I could replicate those results. The sending device is a Google Nexus 6P on T-Mobile using Silence as the SMS app. The receiving device is a Samsung Galaxy S7 on Verizon using the default SMS app that ships with the phone. The tests were performed on 18 November 2016 and both devices were located in Arizona.

Sending 7-bit encoded messages had the same results. I have no way of evaluating the claims of the person who sent me the email. It could be that there are regional issues, and it works in some places and not in others. It could also be that it used to work in 2015 but Verizon has done something to break compatibility since then. My initial testing was performed in June of 2016, which correlates well to this report that Verizon had recently broken inbound concatenated SMS support. In all the testing I have done, I have yet to see a concatenated SMS be successfully received by a Verizon device when sent from another carrier.

Thanks to Roger Pollard, Linda Stoutner, and the many other people who assisted with testing but declined to be listed by name on the website.