Networking Problem

Does anyone encountered out of order packets when transmitting packets of information using SharedEvents?

Currently, I am doing a project that requires to transfer information from between clients. It is similar to transport protocol that takes care of process to process communications. Given a message, I split it into segments and distribute to the clients through SharedEvents.

I did some simple test and confirmed that the receiver will receive all the segments just that occasionally the segments will arrive out of order at the receiver's side. Therefore this means that I have to resequence the segments into proper order and join them up to form the original message before dispatching it for further process. However I have really no idea of how to do it. Does anyone have any idea of solving this problem?

Btw, I am using vrmlscript for the programming part and the technology platform I am working on is blaxxun communication server.

hello milostylo ,
welcome

I think that 2 or 3 guys here will be able to answer ,
I asked them to read the question ,

see you soon ...I go to my school for the moment

Hi milostylo !

I'm neither an expert in the Blaxxun Plattform nor in SharedEvents but I love messing arround with script and servers and vrml and your question sounds pretty interesting to me and we will learn a lot with it.

Probably there will be soon half a dozen of old Blaxxunians able to answer your question.

May be it's just a matter of indexing your packets or giving them a timestamp so they can be sorted in the needed order by the client.

But how could we include such an index or timestamp to be read by the client? What are these packets transporting?

The number of packets one message is split into ... is it allways the same?

Why do you need to split them?

It will need that you show part of your code...

Good luck with it and welcome !

Peter

Hi Peter,

The packets in my program are transmitting text strings. Inside these text strings could contain information like the coordinates of VRML objects, inlines or even chunks of VRML codes.

It is possible to add an index or timestamp. At the sender side, the program will append the index onto the message to be sent. The receiver side will have to know how to separate the index from the message. There is a size limit for each segment. If the size limit is exceeded, a new segment will be generated.

As I have mentioned earlier, I did a simple test and found out that the segments only arrived out of order. No packets are lost during the transmission for segments that contain less than around 1000 characters. But for segments that contain more than 1000 words the probability of information loss is higher. That is why I segment the information.

Here is part of my code that handles the segmenting and transmission of messages:

// Segmentation of message
function send(mm){
var pos = 0;
var len = mm.length;

while (pos < len)
{
var end = len;
if (len - pos > 128){ end = pos + 128; }
sendingMessages[sendingMessages.length] = mm.substring(pos,end);
pos = end;
}

if (msgUser == '')
{
//Compete for the lock
setMsgLock = 'REQUEST';
}
}

// Transmission of message
function sendOneMsg(tt)
{
sendTimer.enabled = FALSE; var header = ':' + sendingMessages.length + ':' + seqNumber + ':';
var message = new String(sendingMessages[sendingIndex++]);
seqNumber++;

if (sendingMessages.length == sendingIndex)
{
seqNumber = 0;
sendingIndex = 0;
prev = 0;
sendingMessages = new MFString();
}
sendMessage = '@' + header + message + '#';
}

hum , I can't help

but I asked Thyme ( I guess you know him , milo ) , to read the topic

hi milostylo alain and peter

Interesting to hear No packets are lost during the transmission for segments that contain less than around 1000 characters

I would simply try making each one of your strings/packets begin with a number (converted to text) so that each string can be ordered by the receiving client no matter what order they arrive.

for example (if packets were only alowed to be 5 characters long not including the index):

"the cat sat on the mat"

would be sent as:

Code:

"0_the c"
"1_at sa"
"2_t on "
"3_the m"
"4_at"

the underbar character defines the end of the text used for the index.
a MFString would make it simple to order the strings back to the correct order using the index (converted to a number)

After all of the packets are added to the MFString the strings can all be joined to make a single string.

for example in the following code packets is a MFString and sentence is a SFString

Code:

for (i = 0; i < packets.length; i++){
    sentence += packets[i];
}

if you need help converting the index text to actual numbers I can write a simple example to do this.

Hope I understood the problem, let me know if I missed the point

Hi milostylo :

Given the fact that you need to split the messages and that the order of arrival could be messed up, I would try to work out a solutioin for restoring the original order of the packets at the client side.

The needed information is allready being sent within the header as sendingMessages.length and seqNumber. I don't know how far the header is intended or able to arrive up to the vrmlscript in the client or not. (may be it's just for the server ).

In my -> VRML way of thinking - which might not be usefull in this case - or if the header can't be read out by the script, I would join again seqNumber and sendingMessages.length in a string with the message itself, and split it back at the client side.

Evidently your seqNumber and sending.Messages.length will be STRING at that point, so need to be converted back into NUMBER.

Knowing sendingMessages.length, you know when you've got all packets and can start to order by seqNumber.

Just tell me if your problem is nothing related with all that, and sorry in that case.


Peter

Code:

Peter's code acknowledges Thyme's existence.

Sorry, I was writing my message while Thyme allready had answered.

Thanks Thyme for your feedback.

Your solution would work if MFString is allowed to be sent through the network. However blaxxun's documentation stated that due to network capacity reasons, it is only possible to send SF~ data via shared event.

Therefore I can only add the text string that is to be sent onto a SFString.

Hi milostylo :

As far as I understood :

thyme wrote:

for example in the following code packets is a MFString and sentence is a SFString

var sentence; // added

for (i = 0; i < packets.length; i++){
sentence += packets[i];
}

sentence is still an SFString, ready to be sent ... ... you've overseen that.

Anyways, you may also join an array (MFString) at the clientside to a string (SFString) using join() in ecma/vrmlScript somehow like that:

Code:

sentence=components.join("%");

or implode() or somesuch at the serverside (I'm talking now, but for sure it does exist in other server scripting languages in a similar way):

Code:

$sentence=implode("%",$components);

Once the receipient gets the SFString (sentence), it needs to be split into it's components, using :

Code:

component=sentence.split('%');

It returns an array, ergo : an MFString ... sent through the network.

Now your data might be decoded as

Code:

var packets_expected=ccomponent[0];
var packet_number=component[1];
var message=component[2];

So if the first packet you receive, out of a yet undefined amount of packets to be expected, contains a string like "7%3%my taylor is rich%", split('%') returns an array('7','3','my taylor is rich').

Unfortunately, this array is a string array, so '7' and '3' aint valid numbers. I don't remember if vrmlscript knows how to convert a string into a number using ecmascript's Number().

I once had to workarround vrmlscript (Blaxx subset) for having charCodes turned into characters, and it was a mess, but I guess that Number() is implemented. If it is not ... Thyme will tell us a workarround

Cheers,

Peter

I am able to implement what you have mentioned below.
My main concern now is the resequencing of the packets. As the packets will arrive at each client differently due to network delay, I need some sort of algo to resequence the packets into proper order and join it up together.

It is like TCP but the problem is the sender doesn't receive acknowledgement from the receiver. The sender will send all the packets without the receiver's acknowledgement.

Still cracking my head over this major issue....

btw, does anyone know whether it is possible to reroute an event back to the function itself?

E.g
eventIn SFString Receive
eventOut SFString Send

function Receive(ss)
{
Send = ss;
}

ROUTE Send TO Receive;

I tried but it dun seems to work. Did I do something wrongly?

Hi milostylo :

The routing from a function to itself should work the same as calling a function from within itself... under some circumstances it does, but if it results in an endless loop or it slows down the execution of other threads it won't.

Your example results in an endless loop, same as this one

Code:

DEF   myScript Script   {

   eventIn   SFInt32 Receive
   eventOut SFInt32 Send

   field SFInt32 counter 0


   url   "javascript:
   
   function Receive(_SFI){
   
      print('counter='+counter);
      Send=counter;
      counter++;
      if(counter>-1){Receive(counter);} // is allways true
   
   }

   function initialize(){
   
      Receive(0);
   
   }
   
   "
}

Blaxxun does not even open the browser.
Cortona stops executing the script after some 998 iterations.

Just changing to this, makes it run on both (the loop is finite) :

Code:

      if(counter<10){Receive(counter);}

Same aplies if you use ROUTE instead of the function call.

Code:

ROUTE myScript.Send TO myScript.Receive

For your Networking problem, I can hardly know if this is exactly your question about :

We split our message "vrml loves blaxx" into 3 substrings

msg[0]="vrml l";
msg[1]="oves ";
msg[2]="blaxx";

We add to each part of the message an INT telling how many packets the current message consists of (3 in this case) and the index of each part (0,1, 2).

Could be like that :

Code:

packet=new MFString();

for(ii=0; ii<msg.length;ii++){

        packet[ii]=msg.length+'%'+ii+'%'+msg[ii];

}

You send each of the packets ... and don't know in what order they will arrive to the receiver.

At the receiver's side, I would have a field in the function, set to the number of packets still to be expected, an eventIn for receiving each packet and an MFString containing the packets as they arrive. The eventOut is optional :

Code:

field SFInt32 packets2go 0
field MFString in_message []

eventIn SFString incomming_packet

eventOut SFString out_message

The function would look like that :

Code:

function incomming_packet(_SFS){

      packet=_SFS.split('%');

      if(packets2go==0){
           
            message.length=0;
            packets2go=Number(packet[0]);

            for(ii=0;ii<packets2go;ii++){
                 
                    message[ii]="";

          }
      }
     
      message[Number(packet[1])]=packet[2];
      packets2go=packets2go-1;

      if(packets2go==0){

          out_message=in_message.join();
    }
     
}

... isn't it at least an aproximative sketch of what you need?


If I still did not understand the problem, sorry. If you guess that I will never understand the problem, just tell me to stop spamming this thread

LOOL Peter )

Quote :

If you guess that I will never understand the problem, just tell me to stop spamming this thread

don't worry ,
I am better than you to bore people , look :


I wanted since days , to ask milostylo ,
if he accepts and has time ,

to tell us who he is , where we could look at his vrml ,
etc ...


scuse , milostylo , I am not in the cop company , LOL ,
just curious of course

and sorry , I can't help on techies stuff ,

My code above, about the Networking thing is complete BS, because neither Number() nor split() nor join() seem to be supported by vrmlscript (at least Blaxxun's).

It will need a workarround with a lot of substring and indexOf ....

Anyways ... it's a sketch.