Page 2 of 2

Re: Lostgallifreyan’s ORG-Link

Posted: Tue Feb 07, 2023 9:43 pm
by Lostgallifreyan
Think of 'Serenity'. The engine is not pretty, but it will keep flying.

Re: Lostgallifreyan’s ORG-Link

Posted: Sun Feb 12, 2023 11:54 am
by Zenerdiode
Lostgallifreyan wrote: Mon Feb 06, 2023 11:53 am …because Psion protocol… I think it's a nice protocol, though awkward to parse at times.
You’re not kidding. From what I’ve seen of the Psion protocol, it’s very similar to one that is used for comms between two industrial processors where I work. I’m writing a Windows based device for the human interface end; and it uses STX and ETX to top and tail the payload - with a checksum at the end. I suspect the bit you say is awkward to parse, is where you could have bytes in the payload that are the same as STX or ETX.

In the system I’m working on, you have to precede a STX or ETX with DLE (data link escape - $10) also a byte $10 with DLE - that’s where it twisted my pea brain. I cracked it though, with some flags to test if it was the first $10 etc. well, obviously you know the drill!

I admire coders like yourself, because sometimes I can sit and watch the flashing cursor for hours, knowing what I want it to do, but nothing comes out of my fingers. Other times I have re-visited my code and thought Wow! I can’t believe I actually wrote that.

I’m terrible at commenting my code, though…

Re: Lostgallifreyan’s ORG-Link

Posted: Thu Feb 16, 2023 12:35 am
by Lostgallifreyan
I used to be terrible at commenting code (I was lousy at taking notes in school too). People often say that the expert should write the manual. This is nonsense. Did the obsessive geographers draw the maps of the world? Obviously not, it was the first explorers who had to start doing it. If everyone waited for 'experts' we'd never have any maps! :)

About the 0x10 bytes, with Psion protocol they get doubled as 0x1010 for travel, then it's easier to know where packets start and end, because their top-and-tail use of 0x10 is not doubled. Those industrial processors, were they using MODBUS? It seems extremely likely, and I think the packet forming rules might be similar. The 16-bit CRC method is certainly the same, but uses a different starting seed, -1 I think, instead of 0 as used in Psion CRC's. Using non-zero seed is better, it prevents a risk of same result when the data buffer starts with some number of zeros.

I don't know how other coders handle the start-of-packet recognition, but I solved it by deduction, and the result would work for any search for a substring too, not least in cases where batches of disk sectors, or 16-bit-addressed RAM data segments, had to be searched, so the boundaries became invisible, the data appearing like a sequential stream in a byte-by-byte read, even when more bytes got read from store or serial port at a time. That really helps for packets, because the software has no control of what comes in next. This way, there are no 'false positives', and spurious bytes are always safely rejected and thrown away. It's usually faster than using an existing library function, and unlike those, I can explore and change stuff at need. Better control in general..

I had to look up the STX, ETX, DLE terminology in a list of control chars I have here. It makes sense, though I didn't think of that when writing ORG-Link! It never occured to me as anything other than a pattern that allowed unambiguous detection of a packet holding any binary content. That is curious, given that the intent of STX and ETX was to mark start and end of text! It is this efficient means of making it store binary data that impressed me. It's probably what eventually evolved into the TCP protocol, which uses similar methods, including the 'sequence number' to determine if all packets arrived in the right order. I sometimes wonder if Psion protocol embedded in UDP might be faster for file transfers. It wouldn't need to use the Psion checksum because UDP uses one. The sequence-and-ACK method of Psion's system could make UDP as reliable as TCP. The main question is whether there'd really be an advantage. Given that data may have arbitrary numbers of 0x10 bytes in the payload, only real-world extensive testing could prove a case for it. If it did fail the test, it may be UDP's minimum datagram size rather than what Psion's protocol can add, that would let it down. For bulk data, it might win. There is talk of using UDP for browsers, which I think is a stupid idea, but it might work as a basis of reinventing FTP, which is far more important than browser makers with stupid ideas want us to believe!