Posts Tagged ‘socket’

Fixed IPV6 accept connection problem.

Beginning with Chilkat v9.4.0, two new TCP socket performance properties, SoSndBuf and SoRcvBuf, will be added to each of the following Chilkat components/libs: FTP2, HTTP, SFTP, SSH, SshTunnel, IMAP, and MailMan (POP3/SMTP).

These properties allow for the underlying socket’s send and receive buffer sizes to be set. These are socket options associated with the setsockopt system call (see http://linux.die.net/man/2/setsockopt ) using the SO_SNDBUF and SO_RCVBUF options.

It is recommended that these socket options remain unchanged unless performance seems slow. If uploads seems slow, then set the SoSndBuf to a larger buffer size. It is recommended to use buffer sizes that are a multiple of 4096. The default buffer size varies from system to system. To get the value, examine the LastErrorText property after calling any Chilkat method that establishes a TCP connection. The socket’s send and receive buffer sizes will be reported.

Likewise, if downloads seem slow, set the SoRcvBuf to a larger value.

For more information about TCP performance, see these web pages:

http://onlamp.com/pub/a/onlamp/2005/11/17/tcp_tuning.html

http://smallvoid.com/article/winnt-winsock-buffer.html

http://www.ibm.com/developerworks/linux/library/l-hisock/index.html

If the LastErrorText contains a line such as this:

socketError: Permission denied

It means you need to add permissions for Internet communications in your manifest. It should be placed outside of the application tag, such as:

<manifest>
    <application>
     .
     .
     .
    </application>
    <uses-permission android:name="android.permission.INTERNET" />
</manifest>

One possible cause of this socket connect error is when an Anti-Virus program blocks the connection. This can happen if the AV program is blocking the outbound port. If some programs can connect to a given remote host:port, but your application cannot, then check your Anti-Virus program to see if exceptions were made for specific programs, but not for your application’s EXE.

If you’re just starting to program with TCP connected sockets using the Chilkat Socket API, then these concepts should be understood before beginning.

1. Receiving Data from a Connected Socket.

The ReceiveBytes and ReceiveString methods will return whatever data has already arrived and is available on the connected socket.  It is not guaranteed to return the complete amount of data that may have been sent by the server.  It may require multiple calls to receive whatever was sent by the server, and your application must know when it has received the full amount.

2. Knowing How Much Data is to be Arriving on a Socket.

When two programs communicate by sending and receiving data over connected sockets, the receiver must know how much data is forthcoming (so that it knows when it has a full “message” — otherwise the receiver will continue trying to read the socket and will “hang” because nothing more is forthcoming..)   There are three ways of accomplishing this:

  1. Use pre-defined packet lengths. In this case, the connected peers have pre-agreed upon a fixed-size amount of data that will constitute one “message”.  For example, if a message is pre-agreed to be exactly 1024 bytes, the receiver knows that it must read exactly 1024 bytes and once fully received, it can stop reading and process the “message”.
  2. Prefix data with a byte-count. The connected peers could use a simple technique such that each message is defined as a fixed-length byte count followed by the message.  For example, if messages are to always be less than 256 bytes, one might communicate by sending a 1-byte integer length followed by the message data.  If messages are to be longer, one might send (perhaps) a 4-byte integer length followed by the message data.  If a multiple-byte integer length is sent, the byte-order (little-endian or big-endian) must be pre-agreed upon.   The Chilkat Socket API provides methods to help implement this technique:  SendCount, ReceiveCount, and ReceiveBytesN.
  3. Read until end-of-message marker is seen. This technique is commonly used by many Internet protocols.  The idea is that the end of a message is defined to be a pre-agreed upon sequence of bytes or string.  For example, CRLF (carriage-return, line-feed) is commonly used in protocols such as FTP, POP3, SMTP, etc.    The receiver continues reading the socket until it receives the sequence of bytes that indicate the end-of-message.  The Chilkat Socket API provides methods to help implement this technique:  ReceiveUntilMatch, ReceiveUntilByte, ReceiveToCRLF, and ReceiveStringUntilByte.

3. Sending Data on a Socket.

When data is sent on a connection via any of the Chilkat Socket methods, what really happens is that the data is passed to the Windows OS for buffered output on the TCP connection.  (This is the same for all socket programming APIs, whether it is Chilkat or non-Chilkat.)  You can imagine that if an application sends a lot of data in a rapid succession of SendBytes method calls, the initial calls will return instantaneously because the Windows OS has plenty of space in the outgoing buffers.  If the application continues to send more data faster than the data can actually be sent over the Internet connection (perhaps because the connection is slow, or the peer is slow to process incoming data), then eventually the SendBytes method will “hang” until enough space is available in the outgoing buffers.  This is the reason for the existence of a MaxSendIdleMs property.  It is also the reason you’ll see throughput spike at the beginning of a data transfer and then slowly decrease to the steady-state throughput rate of the connection.

4. Making a Connection

Initiating a connection with a remote peer (or server) is like making a telephone call.  The attempts to connect to a remote endpoint specified by an IP address and port number.  (If a domain name is used, it is internally converted to an IP address via a DNS lookup.)  An application calls Connect and then waits for an answer.  The maximum amount of time to wait is specified in the last argument to the Connect method.  The ConnectFailReason property indicates the reason for failure.  If the Connect fails because of a timeout, it is due to one of the following reasons:

  1. A firewall at either the client or server side is blocking the connection.
  2. There is no server listening at the remote host:port to accept the connection
  3. Some other software, such as an anti-virus or anti-spyware program is blocking the connection.
  4. The server was too slow in accepting the connection.

It is impossible to know which is the reason because just like a telephone call, the only information you have is that it wasn’t answered.

5. Sending and Receiving Strings

Let’s say you want to send the string “ABC” to the connected peer.   Most programmers implicitly assume it means sending three bytes:  0x41, 0x42, and 0x43.   This is usually correct, but the assumption was made that the communicating programs have pre-agreed upon using the ANSI charset (such as Windows-1252)  such that “A” is represented by a single byte having the value 0x41, and “B” is represented by 0x42, etc.    What if the sender sends ANSI but the receiver is expecting Unicode (ucs-2).  The receiver would be expecting 6 bytes because “A” is represented by 2 bytes (0x41, 0x00), “B” is represented by 0x42, 0x00, and so on..

The StringCharset property controls how strings are sent and received.  It defaults to “ANSI”.  If, for example, it is set to “Unicode”, then a call to SendString(“ABC”) would result in the sending of 6 bytes:  0x41, 0x00, 0x42, 0x00, 0x43, 0x00.   The ReceiveString method would know to interpret the incoming bytes as 2-byte/char Unicode chars and correctly return the string “ABC” to the caller.

This is even more crucial when sending non-English characters, where there are many more choices for character encodings.  For example, Japanese strings might be sent in Shift_JIS, iso-2022-jp, utf-8, Unicode (ucs-2), euc-jp, etc.  It is important that the communicating peers agree on the byte representation of the strings sent and received.

Question:

Can you tell from this log why it would say “socket is not ready for writing”?

  OpenSmtpConnection:
     DllDate: Jun  9 2009
     UnlockPrefix: ****
     Username: Administrator
     Component: ActiveX
     Need new SMTP connection
     SMTP_Connect:
       Connecting to SMTP server mail.****.com:25
       smtp_host: mail.****.com
       smtp_port: 25
       domain: mail.****.com
       smtp_user: ****
       socket is not ready for writing
       Connect function failed.
       SocketError: WSAEWOULDBLOCK The socket would block.
       For more information see http://www.chilkatsoft.com/p/p_172.asp
       Failed to connect to SMTP server.
     Failed to connect to SMTP server

Answer:

All programs running on Windows, Linux, Unix, etc. that communicate using TCP/IP sockets ultimately use (directly or indirectly) the socket system calls (WinSock for Windows, but the functions are virtually the same). These include the following system-level functions: bind, listen, accept, connect, recv, send, select, etc. (see Guide to Network Programming) Even if your coding in Java or .NET, internal to the Java Runtime or the .NET Framework, it is these same socket-level functions that are ultimately called to carry out the work of connecting, reading, and writing TCP/IP sockets. It’s been that way for 30 years…

Naturally, the Chilkat libs call the WinSock functions to implement all of the TCP/IP based protocols — SMTP, IMAP, POP3, FTP, HTTP, SSH, etc. Internally, Chilkat uses non-blocking sockets — meaning that we never want to “hang” the calling thread. This is accomplished internally by using the “select” system call. Chilkat never reads, writes, connects, or tries to accept a connection without first knowing that the socket has data waiting, or that the socket function call will complete without delay. If you examine the description of the “select” WinSock function, you’ll see that there are arguments for both read and write file descriptors. Here’s an excerpt:

...
"The parameter writefds identifies the sockets that are to be checked for writability. If a socket is 
processing a connect call (nonblocking), a socket is writeable if the connection establishment 
successfully completes. If the socket is not processing a connect call, writability means a send, 
sendto, or WSASendto are guaranteed to succeed."
...

The amount of time your application is willing to wait for the Connect to complete is controlled by
setting the Chilkat ConnectTimeout property. If no connection is possible in this amount of time,
(internally) the “select” function times out — i.e. the socket is not “writeable”. We get the
WSAEWOULDBLOCK error because technically, if we called the socket “connect” system call — it
would block (i.e. your program would hang).

A failure to connect is caused by one of the following:
1) There is no server listening at the remote hostname:port.
2) A firewall at either the client or server side is blocking the connection.
3) Something else is blocking the connection — perhaps TCP/IP port filtering or anti-virus.
4) The server is too slow to respond — perhaps your ConnectTimeout is too short and the server is far too busy…

Question:

I need to send a unicode string (e.g TCHAR *ptr) but the API only allows to send char.

Answer:

/*
The _TCHAR data type is defined conditionally in Tchar.h. 
If the symbol _UNICODE is defined for your build, _TCHAR is defined as wchar_t; 
otherwise, for single-byte and MBCS builds, it is defined as char.
*/
bool sendString(CkSocket &sock, TCHAR *str)
    {
    bool success = false;

#ifdef _UNICODE
    success = sock.SendBytes((const unsigned char *)str, _tcslen(str) * 2);
#else
    success = sock.SendString(str);
#endif

    return success;
    }

NOTE: If this error occurred while trying to establish an FTP data connection, also see this: http://www.cknotes.com/?p=282

A WSAEWOULDBLOCK error when trying to establish a TCP/IP socket connection indicates one of the following conditions:

  1. A firewall at either the client or server side is blocking the connection.
  2. There is no server listening at the remote host:port to accept the connection
  3. Some other software, such as an anti-virus or anti-spyware program is blocking the connection.
  4. The server was too slow in accepting the connection.  Increase the value of the ConnectTimeout property.

You may open a DOS prompt and try to telnet to the remote host:port to test connectivity. This page provides more information: WSAEWOULDBLOCK

Note: It is impossible for the client to distinguish between any of the cases described above.  It’s similar to if you were knocking on a door and nobody answers — you have no information — you don’t know if the person simply chose not to answer the door, if the person is not home, or if the person simply cannot hear the knocking.  In all cases, the client’s knowledge is the same: there is no response and you have to decide how long you’re willing to wait…

This blog post is an attempt to explain the concepts of asynchronous socket programming using the Chilkat Socket class/component.   There are five types of socket operations that may occur asynchronously:

  1. Socket read.
  2. Socket write.
  3. Connect to remote hostname:port
  4. Accept connection from client
  5. DNS lookup

A synchronous socket operation is easy to understand.  If you call ReceiveBytes, the method returns only after it has received data on the socket.  If you call Connect, the method returns only after a connection has been established.  If you call AcceptNextConnection, the method returns after an incoming connection has arrived and has been accepted.

Asynchronous methods allow your application to start a socket operation in a background thread.  Your application is then free to do other things while the background thread is working on the socket operation.  For example, your application might do database queries, or update the user interface, etc.  Your application can periodically check to see if the asynchronous operation has completed.  It does this by checking a component property such as AsyncReceivedFinished, AsyncSendFinished, AsyncAcceptFinished, etc.  When the socket operation is finished, your application can get the status via properties such as AsyncReceiveSuccess, AsyncSendSuccess, etc.  Other properties provide the data recieved or the socket object for an accepted connection.

IMPORTANT:  The purpose of the asynchronous socket methods is NOT for queueing up a bunch of socket operations.  For example, if your application calls AsyncReceiveString to initiate a socket receive operation in the background, you cannot call AsyncReceiveString again until the first asynchronous receive completes.   But think about it:  Why would you have more than one asynchronous read oustanding at a time on the same socket?  It doesn’t really make sense.  If Chilkat were to automatically start another read, where would the data go?  When an AsyncReceiveString completes, the received string is available in the AsyncReceivedString property.  Your application can capture or use the received data and initiate the next asynchronous read.  (You may say: “But I want my application to continue reading for performance reasons.”  The answer is:  The operating system is already doing that for you in the background.  The operating system manages connected sockets.  When you receive data on a socket, in most cases you’re simply getting the data that the underlying operating system has already received and has waiting for the application in the underlying socket buffers.)

Regarding asynchronous sending:  Don’t forget that the operating system buffers socket operations.  Unless the outgoing OS buffers are full, or the amount of data you’re sending in a single call is very large, a synchronous send will more-or-less return immediately.  (Did you ever notice when using an FTP program such as WS-FTP, or CuteFTP, that when you upload a file to an FTP server, the upload begins very quickly but then slows down?  It’s because the first amounts of data sent on the socket are buffered by the OS.  The “send” completes while the operating system continues to work (in the background) to send the data in the outgoing socket buffer.  The application is free to continue (in other words, the operating system says “thanks, I have the data and I’ll send it, you may continue with what you’re doing”.)  A socket send operation will only block when the underlying socket send buffers are full.

Back to the main point:  Your application cannot start another asynchronous send operation until the outstanding send completes.  The same goes for Connect and Accept operations.

IMPORTANT:  The Chilkat Socket component is full-duplex, which means:  You may have simultaneous asynchronous receive, send, connect, and accept operations.   You cannot start more than one asynchronous operation of the same type.  For example, if you start an asynchronous receive, you may also start an asynchronous send even though the asynchronous read has not yet completed.

Question:
Do you have any suggestions on this error? I have all the service packs installed for server 2003. Do I need to send shorter strings? I don’t seem to have any issues with small messages.

ChilkatLog:
  SendString:
    DllDate: Sep 12 2008
    Username: XYZABC
    Component: .NET 2.0
    NumChars: 502583
    Charset: ansi
    NumBytes: 502583
    SocketError: WSAECONNABORTED An established connection was aborted by the software in your host machine.
    Error sending on socket
    send_size: 65535
    Failed.

Answer:
The word “software” in any of the above WSAECONNABORTED error message refers to “WinSock”. In a nutshell, it’s a logjam. The sending software (i.e. WinSock) is backed up to the point where all internal OS buffers are full and it must abort the connection. The solution is to send smaller amounts, or make sure the receiver on the server-side can consume incoming data faster. Here is a more detailed explanation: WSAECONNABORTED