Home › ELEKTOR FORUMS › DigiButler (April & May 2008) › HTTP server does not like IE ?

ELEKTOR FORUMS

Please log in to post a reply or subscribe / unsubscribe to topics

Topic: HTTP server does not like IE ?

Author Post

ponedelnik

10 posts

Popping In
Popping In

Read post 26-08-2011 11:57

The compiled web in the Digibutler software includes the page "variables.htm", which reloads itself every second or so. I observed that with Internet Explorer (IE) this process would stop after some unpredictable time, whereas with Firefox it would usually survive for days.
After a stop, the server would still be alife and accept a new request.
So, how could it tell IE from Firefox ?
The only difference I could think of was the contents of the request the user agent (a technical name for the navigator) was sending to the server, a collection of data called a header.

A header is made of a number of lines, each with the syntax :

data_name: data_value

except for the first and the last lines.
The last line is blank and is used to terminate the header.
The first line specifies the access method, the name of the requested file and the protocol :

GET /variables.htm HTTP/1.1

That is the only line that's useful to our little server.

A typical header sent by IE is as following :

GET /variables.htm HTTP/1.1 Accept: image/jpeg, application/x-ms-application, image/gif, application/xaml+xml, image/pjpeg, application/x-ms-xbap, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword,
Accept-Language: fr-FR User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.507
27; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0;.NET4.0C) Accept-Encoding: gzip, deflate
Host: 192.168.1.59
Connection: Keep-Alive

The size of this one is 474 bytes, and it is probably not the largest !

Now let's look within module "freescale_http.c" at function "freescale_http_read_header(…)", where the received header is read with the code :

length = m_recv( freescale_http_sessions[session].socket, (char *)buffer, RECV_BUFFER_SIZE );

The size of the receive buffer is defined within module "freescale_http_server.h" :

//*****************************************
// Receive buffer, DO NOT CHANGE!!!
//*****************************************
#define RECV_BUFFER_SIZE	0x100

This is 256 bytes, much smaller than the received 474 bytes.

This means that the TCP/IP stack will keep the rest and return it on the next call of m_receiv().
If this call is assumed to return a new header, we'll process garbage, which we may detect as such or not (in the best case we can't satisfy an invalid request and return a "page not found" error, effectively creating the condition we started with), therefore we should discard the data in excess by calling m_receiv() repeatedly until the end-of-header line is found.
But m_receiv() has got to copy the data to some buffer. Of course, we could allocate a buffer from the stack, but this would increase the usage of the stack, not the best solution.
If we accept to limit the size of the file name (including ?forms) to a manageable 160 characters, the first header line would never exceed 192 bytes, so we could use the last 25% of the receive buffer to throw away the garbage.

The modified function "freescale_http_read_header(…)" would be as following :


void freescale_http_read_header( int session, char *buffer )
{
  HEADER_TYPES	i;
  int	length, j, k;

if(session>MAX_NUMBER_OF_SESSIONS){
  freescale_http_sessions[session].state = EMG_HTTP_STATE_CLOSE;
  return;
}
  if( freescale_http_sessions[session].keep_alive  )
    freescale_http_sessions[session].keep_alive--;

  // PARAM1: M_SOCK socket,
  // PARAM2: char * buffer
  // PARAM3: unsigned length
  //
  // RETURNS: number of bytes actually read, or -1 if error.
    // int m_recv(M_SOCK so, char * buf, unsigned buflen)	// commented out - HM 23-aug-11
  length = m_recv( freescale_http_sessions[session].socket, buffer, RECV_BUFFER_SIZE );

      if( length < 0 )
  {
//    if( !freescale_http_sessions[session].keep_alive )
      freescale_http_sessions[session].state = EMG_HTTP_STATE_CLOSE;
    return;
  }

// ******************************************************************************
// Discard header data in excess
//	As the headers supplied by the user agents tend to inflate, they have overgrown
//	the size of the receive buffer. Only the first header line is of interest to us,
//	the rest can safely be discarded.
//	A scratch zone at the end of the receive buffer is used to store the header data
//	in excess. It is overwritten by calls to m_recv until the returned data is
//	shorter than expected or is terminated by a blank line (double CR/LF).
//	Note : it is assumed that the useful data does never overflow into the
//	scratch zone. This limits the size of the scratch zone, which should be large
//	to limit processing overhead.

#define EXCESS (RECV_BUFFER_SIZE/4)		// size of scratch zone at end of buffer

  if (length >= RECV_BUFFER_SIZE)		// if smaller, there is no excess data
    do {
//	The following test takes care of the possibility that the returned data fills the
//	the scratch zone exactly, meaning there is no more data to receive.
      if ((buffer[RECV_BUFFER_SIZE-4] == 13) &&
        (buffer[RECV_BUFFER_SIZE-3] == 10) &&
        (buffer[RECV_BUFFER_SIZE-2] == 13) &&
        (buffer[RECV_BUFFER_SIZE-1] == 10)) break;
    } while (EXCESS == m_recv( freescale_http_sessions[session].socket,
        &buffer[RECV_BUFFER_SIZE-EXCESS], EXCESS));
// ******************************************************************************

  // Scan through the buffer looking for the method
  for( i=NO_HEADER_FOUND; header_table.header_string[0]; i++ )
  {
    if(i > 3)	/* empty or invalid header */
    {
        freescale_http_sessions[session].state = EMG_HTTP_STATE_CLOSE;
        return;
    }

    for( j=0, k=0; j < 8; j++ )	// the name of the method must be within the 7 first chars
    {
      if( buffer[j] == header_table.header_string[k] )
        k++;

              if( header_table.header_string[k] == 0 )
      {
#if HTTP_VERBOSE>3
        printf( "\n6 header[%d] = %s", session, &header_table.header_string[0] );
#endif
        freescale_http_sessions[session].headertype = header_table.header_type;
 //       header = freescale_http_sessions[session].headertype;

#if 0
        // Scan to /
        for( ; j<length; j++ )
        {
          if( buffer[j] == '/' )
          {
            freescale_process_header( session , &buffer[++j], length-j );
            return;
          }
        }

                        // If we get here, we did not find the '/'
        // Just send the default file
        buffer[0] = ' ';
#endif
        freescale_process_header( session , buffer, length );
        return;
      }
    }
  }

      // If we get here, then
    //	We do not support the method 'GET', 'POST', 'EMG'
  //	The only thing we can do is close the socket
  freescale_http_sessions[session].state = EMG_HTTP_STATE_CLOSE;
}

shorty3

6 posts

Popping In
Popping In

Read post 04-09-2011 17:53

Hello,

I tried your code modification of freescale_http.c, but this leads to an error as in my version the struct headertable is the following:

const HEADER_STRUCTURE header_table[] =
{
...
};


whereas your code tries to access

header_table.header_string[0]  


This leads to the following error message:

Error : not a struct/union/class
freescale_http.c line 611 for( i=NO_HEADER_FOUND; header_table.header_string[0]; i++ )

Maybe I missed another code modification?


regards

Werner

Post edited by shorty3 on 04-09-2011 17:55

Post edited by shorty3 on 04-09-2011 17:57

Post edited by shorty3 on 04-09-2011 17:58

Post edited by shorty3 on 04-09-2011 18:00

Post edited by shorty3 on 04-09-2011 18:01

Post edited by shorty3 on 04-09-2011 18:03

Post edited by shorty3 on 04-09-2011 18:09

ponedelnik

10 posts

Popping In
Popping In

Read post 06-09-2011 23:58

The definition of HEADER_STRUCTURE is as following in freescale_http_server.h :

//*********************************************
// Max size of HTTP command, GET, POST, ...
//*********************************************
#define HEADER_TYPE_SIZE					4

typedef struct
{
	unsigned char	header_string[HEADER_TYPE_SIZE+1];
	HEADER_TYPES	header_type;
} HEADER_STRUCTURE;
The header_table is an array of these structures, therefore, any
access to it must provide a session index:

if ( buffer[j] == header_table[session].header_string[k] )
which does not appear in your quotation.

Hope this helps

Helmut

PS. The text formatter of this forum seems to play strange tricks, (it just silently discards a letter i enclosed in brackets ?)
You may look at my "port to Windows" message for a link to my sources, in case there are other errors of the same kind.

Post edited by ponedelnik on 07-09-2011 00:00

Post edited by ponedelnik on 07-09-2011 00:03

Post edited by ponedelnik on 07-09-2011 00:12

Post edited by ponedelnik on 07-09-2011 00:20

shorty3

6 posts

Popping In
Popping In

Read post 07-09-2011 16:11

Thank you, I will test it...

I already assumed that this might be due to the formatter (I also had to edit my reply several times as it was e.g. not possible to post the entire header structure without strange interaction by the formatter)

...not very useful in a forum, where the posting of source code snippets is important....

regards

Werner

shorty3

6 posts

Popping In
Popping In

Read post 14-09-2011 10:55

Hello,

I tried the suggested modification but without success.

Using the code modification as proposed I could not reach the server at all (neither with firefox nor with IE). After uncommenting the line
if (!freescale_http_session[session].keep_alive) 

(as in the original code) I was at least prompted with the authentication window, when the web-page was requested.
However even after entering the correct authentication values no web page was shown.
Might it be that the for-loop
for (j=0 ,k=0 ;j < 8 ;j++)

causes the problems in case of a zero length of the received header (length = 0)?
In the original code the loop would be bypassed whereas in the modified code the buffer content - which is not valid as no bytes have been received - would be compared to the header-table.

Thank you very much for your help and regards

Werner

Post edited by shorty3 on 14-09-2011 11:20

Post edited by shorty3 on 14-09-2011 13:33

Post edited by shorty3 on 14-09-2011 13:34

ponedelnik

10 posts

Popping In
Popping In

Read post 14-09-2011 23:49

What was it you did as modifications ?
When I wrote :

if ( buffer[j] == header_table[session].header_string[k] )

it was an example, not the actual code because of the formatter problem :

[session] stands here for i enclosed in brackets

May be I should have made this plainly clear.

Personally, I have no interest in authentication and passwords for a little thing I don't expect to be accessed by anybody else, so I disabled these (all the validation or pw manipulation functions are patched to return with a success code). I could well use the freed memory space for more useful application code.

Helmut

shorty3

6 posts

Popping In
Popping In

Read post 15-09-2011 11:11

maybe we had a little misunderstanding here because I was unclear....
with code modification I just meant YOUR modified freescale_http_read_header() function (no problem....!!))

The problem arises indeed if authentication is used, because the subsequent calls of m_recv() in case of an
excess header seem to prevent the correct extraction of the authentication string which is part of the header.

However, when I tested the "original" function yesterday I found out that also using Firefox EVERY call of m_recv() in
freescale_http_read_header() leads to an excess of the receive buffer. Nevertheless, even using authentication,
which extends the header at its end does not lead to problems. At least from that it seems that a header length exceeding the recv_buffer_size is not really a problem in the original freescale_http_read_header() function - or do I overlook something?

regards
Werner

ponedelnik

10 posts

Popping In
Popping In

Read post 15-09-2011 12:13

My understanding (which may be incorrect) of the original code is that when the TCP/IP stack receives a header frame, it stores it away and delivers to the user just as much of it that's wanted by the byte count. If something remains, it is returned on the next read request 'which may not exhaust the remaining data. This subsequent data may be interpreted as a new header and discarded as invalid or not, if by chance it contains some expected data, like a valid method (which search I limited to the 8 first chars of the buffer).
I suppose that theTCP/IP stack itself could experience an overflow, which I did not investigate.

Helmut

Please log in to post a reply or subscribe / unsubscribe to topics

Follow Elektor

      

Act now!

Bestseller

Microprocessor Design using Verilog HDL

This book is a practical guide to processor design in the real world. It presents the Verilog HDL in an easily digestible fashion and serves as a thorough introduction about reducing a computer architecture and instruction set to practice. 

Package Deals

Elektor Bundles

Check our Package Deals and save money! Discounts up to 19% now available!

Price Slashed

Elektor OSPV

This Open Source People Vehicle is perfect for factory halls, warehouses, hospitals, colleges, schools etc.

Elektor PCBs at 25% OFF