Socket Programming – “Must-Know” Concepts
If you’re just starting to program with TCP connected sockets using the Chilkat Socket API, then these concepts should be understood before beginning.
1. Receiving Data from a Connected Socket.
The ReceiveBytes and ReceiveString methods will return whatever data has already arrived and is available on the connected socket. It is not guaranteed to return the complete amount of data that may have been sent by the server. It may require multiple calls to receive whatever was sent by the server, and your application must know when it has received the full amount.
2. Knowing How Much Data is to be Arriving on a Socket.
When two programs communicate by sending and receiving data over connected sockets, the receiver must know how much data is forthcoming (so that it knows when it has a full “message” — otherwise the receiver will continue trying to read the socket and will “hang” because nothing more is forthcoming..) There are three ways of accomplishing this:
- Use pre-defined packet lengths. In this case, the connected peers have pre-agreed upon a fixed-size amount of data that will constitute one “message”. For example, if a message is pre-agreed to be exactly 1024 bytes, the receiver knows that it must read exactly 1024 bytes and once fully received, it can stop reading and process the “message”.
- Prefix data with a byte-count. The connected peers could use a simple technique such that each message is defined as a fixed-length byte count followed by the message. For example, if messages are to always be less than 256 bytes, one might communicate by sending a 1-byte integer length followed by the message data. If messages are to be longer, one might send (perhaps) a 4-byte integer length followed by the message data. If a multiple-byte integer length is sent, the byte-order (little-endian or big-endian) must be pre-agreed upon. The Chilkat Socket API provides methods to help implement this technique: SendCount, ReceiveCount, and ReceiveBytesN.
- Read until end-of-message marker is seen. This technique is commonly used by many Internet protocols. The idea is that the end of a message is defined to be a pre-agreed upon sequence of bytes or string. For example, CRLF (carriage-return, line-feed) is commonly used in protocols such as FTP, POP3, SMTP, etc. The receiver continues reading the socket until it receives the sequence of bytes that indicate the end-of-message. The Chilkat Socket API provides methods to help implement this technique: ReceiveUntilMatch, ReceiveUntilByte, ReceiveToCRLF, and ReceiveStringUntilByte.
3. Sending Data on a Socket.
When data is sent on a connection via any of the Chilkat Socket methods, what really happens is that the data is passed to the Windows OS for buffered output on the TCP connection. (This is the same for all socket programming APIs, whether it is Chilkat or non-Chilkat.) You can imagine that if an application sends a lot of data in a rapid succession of SendBytes method calls, the initial calls will return instantaneously because the Windows OS has plenty of space in the outgoing buffers. If the application continues to send more data faster than the data can actually be sent over the Internet connection (perhaps because the connection is slow, or the peer is slow to process incoming data), then eventually the SendBytes method will “hang” until enough space is available in the outgoing buffers. This is the reason for the existence of a MaxSendIdleMs property. It is also the reason you’ll see throughput spike at the beginning of a data transfer and then slowly decrease to the steady-state throughput rate of the connection.
4. Making a Connection
Initiating a connection with a remote peer (or server) is like making a telephone call. The attempts to connect to a remote endpoint specified by an IP address and port number. (If a domain name is used, it is internally converted to an IP address via a DNS lookup.) An application calls Connect and then waits for an answer. The maximum amount of time to wait is specified in the last argument to the Connect method. The ConnectFailReason property indicates the reason for failure. If the Connect fails because of a timeout, it is due to one of the following reasons:
- A firewall at either the client or server side is blocking the connection.
- There is no server listening at the remote host:port to accept the connection
- Some other software, such as an anti-virus or anti-spyware program is blocking the connection.
- The server was too slow in accepting the connection.
It is impossible to know which is the reason because just like a telephone call, the only information you have is that it wasn’t answered.
5. Sending and Receiving Strings
Let’s say you want to send the string “ABC” to the connected peer. Most programmers implicitly assume it means sending three bytes: 0x41, 0x42, and 0x43. This is usually correct, but the assumption was made that the communicating programs have pre-agreed upon using the ANSI charset (such as Windows-1252) such that “A” is represented by a single byte having the value 0x41, and “B” is represented by 0x42, etc. What if the sender sends ANSI but the receiver is expecting Unicode (ucs-2). The receiver would be expecting 6 bytes because “A” is represented by 2 bytes (0x41, 0x00), “B” is represented by 0x42, 0x00, and so on..
The StringCharset property controls how strings are sent and received. It defaults to “ANSI”. If, for example, it is set to “Unicode”, then a call to SendString(“ABC”) would result in the sending of 6 bytes: 0x41, 0x00, 0x42, 0x00, 0x43, 0x00. The ReceiveString method would know to interpret the incoming bytes as 2-byte/char Unicode chars and correctly return the string “ABC” to the caller.
This is even more crucial when sending non-English characters, where there are many more choices for character encodings. For example, Japanese strings might be sent in Shift_JIS, iso-2022-jp, utf-8, Unicode (ucs-2), euc-jp, etc. It is important that the communicating peers agree on the byte representation of the strings sent and received.