The price of the Internet of Things will be a vague dread of a malicious world. by Marcelo Rinesi. post
Volkswagen didn’t make a faulty car: they programmed it to cheat intelligently. The difference isn’t semantics, it’s game-theoretical and it borders on applied demonology.
The intrinsic challenge to our legal framework is that technical standards have to be precisely defined in order to be fair, but this makes them easy to detect and defeat. They assume a mechanical universe, not one in which objects get their software updated with new lies every time regulatory bodies come up with a new test.
And even if all software were always available, checking it for unwanted behavior would be unfeasible — more often than not, programs fail because the very organizations that made them haven’t or couldn’t make sure it behaved as they intended.
Our experience of the world will increasingly come to reflect our experience of our computers and of the internet itself: full of programs never installed doing unknown things to which we’ve never agreed to benefit companies we’ve never heard of, inefficiently at best and actively malignant at worst.
Swim Test-Case Visualization addressed checking of collaborative software for intended behavior.