Andy Kirkham replied to the topic 'INDI Testing Framework' in the forum. 8 years ago

Gerry

Why do you keep bring up the vendor binary? I keep saying we shouldn't even be loading their binary let alone testing it!

What I am saying is THE CODE YOU WRITE should be tested.

If your driver needs to do some math to move a scope from point A on the sky to point B on the sky I want to know your math is correct and the result of your calculation is a slew command.

Given point A and point B I EXPECT your driver to issue COMMAND foo{...} to the scope. Test complete. That's it.

Now, there maybe bugs in the driver that mean it does something un-expected and you need to adjust the command that is sent to compensate for it. Now that's all this reverse engineering you keep going on about. Personally I would write to the manufacture to get the bug fixed upstream. Failure to fix means no further development and a big public notice on the Indi Website telling people to avoid the buggy product.

Whatever "punishment is deemed fit", that's just an example. So people may prefer the hard life of fixing these buggy vendor binaries, that's entirely down to you. I personally probably wouldn't, I would just do my homework and buy something recommended and known to work in the first place.

When it comes to unit testing I just want to know you wrote the code to the specification you say you are working to, even if that's one adjusted for the failings of manufacturers.

I am in no way advocating fixing commercial stuff in open software, quite the opposite in fact, name and shame them into compliance would be my moto.

What I do expect is OUR CODE to work as we expect. I don't care about vendor binaries. If you do, carry on sweating over it by all means. It's your time.

Unit testing is all about OUR CODE. No one elses.

I really don't understand why you keep ing this issue about vendor binaries at all. They are totally irrelevant to unit testing our own code. Imagine for one second if I manufactured domes and I implemented "DROP TROUSERS" as the serial command to open the roof what I want your driver to emit is that text. All I have to test is the driver emits that. I don't have to stand there waiting to see if the roof opens. I know for a fact my driver is correct and compliant.

If however my driver does nothing despite I said it would open for that command I fully expect an email from you complaining. No response from me gets talked about in social media, in forums, endlessly visible for decades to come to a wealth of future buyers. My reputation plummets and I sell nothing.

But so long as your driver emits the command correctly your unit test will pass. YOU and your code are compliant. That's what unit testing is all about.

This constant talk about having to reverse engineer broken drivers is totally off topic and nothing to do with testing our own code. If they are that bad just don't support them.

Read More...