AI fighter pilot wins in combat simulation
bbc.co.uk
bbc.co.uk
This makes (clever) use of hundreds, if not thousands, man hours of painstakingly entering expert rules if the form IF <some input value is above or below some threshold> THEN <put some output value in the so and so range>.
The mathematical model of Fuzzy Trees is nice, but this is completely ad-hoc to the specific modelization of the problem, and will fail to generalize to any other problem space.
This kind of techniques has some nice properties (its "reasonings" are understandable and thus kind of debuggable and kind of provable, it smoothes some logic rules that would otherwise naively lead to non smooth control, etc.) but despite the advances presented here that seem to make the computation of the model tractable, I don't see how it could make the actual definition of model anywhere near tractable.
Also, I dislike having to wade though multiple pages of advertising before I can find the (very light) scientific content.
-- Edit: I realize I am very negative here. I do not mean to disparage the work done by the authors. It's just that the way it is presented make it sound way more impressive than it is. It's still interesting and novative work.
What I found did not convince me that it would fare better than an off-the-shelf (somewhat) non-interpretable statistical supervised learning algorithm.
It can be a nice way of bootstrapping the rule writing process, or to go the other way : to discover and analyze new expert knowledge by looking at the rules.
But performance-wise, I would go the machine learning way anytime.
Also, Inverse Reinforcement Learning seems to be very promising : one guesses the reward function by observing the expert acting.
For example, a very specific condition can either mean that this particular condition is useful, or that simply the training data happened to have those particular instances of a more general condition - and a human expert can usually easily decide which rules need to be extended for proper generalization beyond your training data, but the automated generation helps identify factors that the expert could recognize, but wouldn't think of if doing it themselves from scratch.
Even outside Buell, which was a bit of a neither-fish-nor-foul anomaly, they've had the odd innovative model here and there--the V-Rod comes to mind. And I've seen some interesting things in their ABS systems and some other components. But I think the most successful models have been very conservative about their drivetrain as it's part of their signature sound/feel.
My hope is that Polaris' recent critical success with the Indian Scout (which I bought over HD--far better bang for buck than a Sportster) gives the segment a kick in the ass.
Buell had some great innovative designs too, but you could tell that was all Erik and not the motor company. I wasn't surprised when the split happened.
The plus side is that it allows you to make systems that can be tweaked using trial and error to handle cases that would require very complex logic otherwise. The down side is that alot of the time the systems aren't provable the way other types of logic are, and they can be a pain to debug.
Well, why should it? No one is inventing HAL-like AI anytime soon, or ever. If this system does a better job of killing the enemy than human pilots then its quite the breakthrough. Projecting air power is one of the ways countries keep aggressors away and this would be quite an advantage for variety of reasons. Not the least of which means you can now design AI driven fighters that have zero design compromises to keep human pilots alive.
I imagine fighter engagement consists of a fairly limited set of problems to solve. Think of this as just a souped up autopilot/autoland system, except with guns and missiles. We're not asking the AI to write the next Romeo and Juliet here.
>because the paper is very light on details
Defense contractors aren't known for sharing details. I imagine this is a competitive advantage and they want to keep their cards close to their chest. There may even be national security issues here.
Because what is trumpeted as a breakthrough may in fact be so narrow in scope that it may not even be possible to use it in a flight combat video game without a lot of work, let alone any real life environment.
It is absolutely, very closely tied to the mathematical model of aerial combat that they devised and can not easily be made to accommodate new insights, or new challenges.
What? They were quite literally running the AI in a simulator -- ie, a very expensive video game. The only thing that might not scale is the computational power necessary to execute.
Learning approaches which depend on mass-simulation are great when your problem only ever exists in a "virtual" context, but what happens when you want to take your trained neural network out into the real world? Clearly it's going to have to adapt to the differences between the real world and the virtual world - but how would you do that? You can't run millions of dogfights in the real world to adjust its training.
?
For small scale problem where most of the variables are well understood, this kind of approaches work beautifully. Big problem are better tackled by a more generic approach (maybe with some ad-hoc adaptations, such as mixed approach between expert systems and statistical algorithms, feature engineering, etc.) because these approaches will be more resilient to an exposure to the real world, and the manpower invested in them is useful in more than one problem domain.
To address your last point, there is an extensive body of work on data-scarce environments. I've even seen a talk about applying reinforcement learning to endangered species preservation, where you only get a single digit number of interaction with the system !
It is very hard to ask for management to trust a system they know nothing about, where they have literally no control over final behaviour, even if in the end it will perform better overall. In a rule-based system, instead, it is always possible to make adjustments and blame mistakes very efficiently to specific causes.
I guess this is the main reason why "true" AI is currently being used mostly in information fields, rather than on physical machines and engineering. No-one would know how to deal with the outcome of a fuzzy learned algorithm making the wrong decision. This is also a reason why autonomous cars are very interesting to me, even though I bet they are still full of ad-hoc rules in order to have a layer of "manageability" over the overall system.
Air combat is also one of those areas which does notionally have narrowly computable victory parameters - given hardware of capabilities X, there is a model we don't know which should generally predict the outcome.
If it ends up like thousands upon thousands of if statements then you got it wrong (or so I think).
Getting it right IMO means realizing that rules are not necessarily source code but often data to be iterated over, filtered etc.
This is a form of metaprogramming that can be practiced even in Java ;-)
But the decision tree is much more tractable, thus the longer it's kept on this format, the more future-proof is the work.
It's still used in a few systems as a complementary system involving PIC controls.
Maybe one day war will be less about killing people, and more of a battle between countries' best engineers.
Maybe I'm just optimistic, but I think robot wars would be a hell of a lot better than real wars.
(That said I'm less optimistic I guess.)
When two robots fight each other, it's not because the other robot is the target, it's because it's an obstacle between them and the human targets that will actually affect the war.
That is of course assuming that the most efficient strategy to achieve a surrender doesn't actually end up being a bypass of enemy AI and direct targeting of human populations. This would only be possible if there's a huge mismatch between offensive and defensive capability. But then you end up with a MAD situation similar to what we have with nukes.
That said, I expect that the flip-side is that this tech would ultimately make a guerilla war by a local population against an occupying force very very ugly for the guerillas.
So it boils down not to robots killing robots, but robots killing humans, while other robots try to stop them.
In simulation, the algorithms demonstrated for doing this do quite well with, being able to complete the mission with a success rate of 97.5%, so long as one has 6 search robots and 3 gun robots.
This did not work so well in real life, partially because real robots are difficult to do with. It is still disturbing though because of the high success rate, not to mention the immediate applicability to robot SWAT teams. As a civilian, I'd be much more concerned with a SWAT AI than a fighter jet AI.
However, robot swat teams are still a ways off.
[0]http://singularityhub.com/2010/03/19/teams-of-military-robot... [1]https://en.wikipedia.org/wiki/Multi_Autonomous_Ground-roboti... [2]http://www.frc.ri.cmu.edu/~ssingh/Sanjiv_Singh/PUBS_CONF_fil...
When an expert system beats some human in a complex real-life problem the comments are about how it is narrow, boring, not sufficiently tested and ultimately doesn't matter.
When a neural network (with the help of MCTS and an entire data-center full of servers) beats some human in a board game the comments here hype it through the roof and jump to conclusions about the coming dawn AGI.
The fighter jet AI technique is hard-coded to a very specific problem domain, and could only be reproduced in a different problem domain by doing it from scratch.
The technique used by AlphaGo is at least closer to the idea that we can eventually build generically trainable machines that can learn to do a variety of tasks, without having to code them from scratch every time.
So are machine learning models, ultimately. Although the algorithms that build the models have more general application, once you train a model on a certain set of data, that model can only ever be used in the domain circumscribed by the data. They're one-trick ponies, yes?
>> we can eventually build generically trainable machines that can learn to do a variety of tasks, without having to code them from scratch every time.
We would however have to train them from scratch every time, for each separate task, and each time we'd need terabytes of data and megawatts of power.
AlphaGo has several handcoded training features, uses Monte-Carlo tree search, and was primed by a huge human-made dataset. It's not like it can be throws at other problems without heave re-engineering.
I think it's because only recently has AI been in the news again and it's been the news thanks to machine learning and neural networks in particular (and more specifically, deep learning). The last time there was a big todo about AI was in the '90s and most people writing here were probably not old enough to figure out what the hell was up back then.
Er, I'm not dating myself here. I know about those things out of happy accident (maybe a story for another time). Most people who graduated from CS in the last five or six years will not have heard anything about expert systems and GOFAI, except that it failed etc, if that.
Then along comes Google and promises it can make your phone talk to you. People are intrigued.
But of course, those who don't know their history are doomed to repeat it.
Human pilots are limited in maneuvering by the amount of force their body can take. Computer pilots are limited by the amount of force the airframe can take. The later allows much more aggressive and radical maneuvers.
That means, compared to playing Go, the robot pilot can be comparatively much worse than the human and still win decisively by taking advantage of high g maneuvers.
In short, it's a hand-crafted database of rules in a format similar to "IF Condition THEN Action" coupled to an inference procedure (or a few different ones).
That sort of thing is called an "expert system" because it's meant to encode the knowledge of experts. Some machine learning algorithms, particularly Decision Tree learners, were proposed as a way to automate this process of elicitation of expert knowledge and the construction of rules from it.
As to the "fuzzy logic" bit, that's a kind of logic where a fact is true or false by degrees. When a threshold is crossed, a fact becomes true (or false) or a rule "fires" and the system changes state, ish.
It all may sound a bit hairy but it's actually a pretty natural way of constructing knowledge-based systems that must implement complex rules. In fact, any programmer who has ever had to code complex business logic into a program has created a de facto expert system, even if they didn't call it that.
For those with a bit of time in their hand, this is a nice intro:
http://www.inf.fu-berlin.de/lehre/SS09/KI/folien/merritt.pdf
What are you failing to understand?
You're right that I didn't. Thanks for correcting me and apologies for the slight fudging. I think my description is still mostly accurate though.
>> I'd guess that they've basically reinvented decision trees.
That would make sense in the sense that it's kind of an obvious algorithm to re-invent if you're trying to learn propositional rules. I'm not so sure about the "evolutionary" part though.
After they completed the training they put it up against real veteran pilots and the AI basically did a few things. It would take evasive maneuvers when fired upon and fire when in optimal range. That's pretty much it. And you know what? That's really all modern pilots need to do. It's amazing what they did with Top Gun, making this stuff not look boring. In the end of the day it's just wait for some computer to tell you that you have target lock and press a button. If attacked, take evasive maneuvers and pray. Takeoff and landing on a Carrier is the scariest part.
I'm quite curious how this system would perform in WWII era dogfights where you had to worry about the stress on your plane, had to deal with engines that failed and stalled all the time and maneuvers that were much slower and closer to the enemy (plus no missiles).
Even so, I enjoyed reading the paper (not the article) so would recommend it if you're into Game AI at all.
Apart from technically superior radar and weapons systems, the USAF Weapons School[0] teaches USAF fighter pilots how to engage within the envelope of those weapons.
In the early part of the Vietnam war, the air-to-air missiles kept missing the target due to poor pilot training. The result of the Ault report[1] and subsequent TOPGUN[2] program worked to remedy the shortcoming in the training. The USAF FWS was created after seeing the results of the Navy TOPGUN training.
[0] https://en.wikipedia.org/wiki/USAF_Weapons_School
[1] https://en.wikipedia.org/wiki/Ault_Report
[2] https://en.wikipedia.org/wiki/United_States_Navy_Strike_Figh...
It makes me wonder a little bit if their skills are vastly overestimated.
Experienced USAF pilots have hit the ground during combat operations. That doesn't mean they were not skilled pilots, just that they made a deadly mistake.
I didn't agree with this at first, but they claim that modern missiles are able to differentiate between heat signature of plane body and engine nozzle. Also modern missiles are capable of 50G turns. That G-loading tells nothing about turning circle, but it does tell surprisingly nicely how quickly you can deviate from straight line. Fighter plane loses every time from every angle at every speed.
The only problem of air-to-air missile dominance is that they burn through all of their fuel relatively quickly. Once that happens, maneuverability goes down really quick. So the "no escape zone" is crucial and it varies from missile to missile.
If you want to stay alive, you need to out-range or surprise the enemy. Either stealth, superior radar or longer range missiles.
It looks like fighters have become very expensive and very mobile SAM sites. It beats me why countries without aircraft carriers would ever pick F35. You can get ~50 trucks with Pirate IRST and METEOR missiles with the same price. More area covered at any given time while survivability goes up like hell I don't know what.
When airplanes get within gun range ("knife-fighting range") things are very interesting. Beyond visual range is just weapons management, but close-in it's energy management to optimize maneuvering to get to a killing position.
Also, it's much more interesting when you have something at stake besides losing a video game and having to restart.
I'm quite curious how this system would perform in WWII era dogfights where you had to worry about the stress on your plane, had to deal with engines that failed and stalled all the time and maneuvers that were much slower and closer to the enemy (plus no missiles).
By the end of WWII, properly maintained engines flown within parameters were pretty reliable. (So no pulling negative g's in a plane with a carburetor.) The concerns of fighter pilots at very close range or trying to evade when targeted by missiles can still be similar in certain regards to pilots trying to stay alive in WWII.
AI's would probably be very good at energy management and taking shots of opportunity.
Recorded conversations between ground support pilots and forward air controllers would disagree. Lots of very fast paced observation, pattern matching, orientation and tough judgment calls.
If we ever fight a competent air adversary I would imagine the AWACS to pilot conversations would be fascinating.
Learning to fly a plane is like learning to throw a baseball. It takes hours at most. Of course learning to beat a pro player at their entire game not just one activity, so as to get their job, is a little harder. An interesting observation of human judgment WRT the ratio of people who think they can go pro vs the people who have the skills to go pro is not terribly inspiring WRT AI pilots, they'll be lots of coders with bravado and not much action. And learning how to lead a team to a world series win isn't even definable at this time. But yeah, toss that ball over there when I say to do it, that's a solved problem. Likewise successfully accomplishing a combat mission is a lot more complicated than "and this is how to keep the wings level and that button makes things go boom". A really smart autopilot is going to help yet isn't the only thing necessary.
Note that its possible to send a man to do a cruise missiles job, or even a plain old missile's job. That doesn't imply a cruise missile can do everything a man can do, it just means the man was mismanaged to not take full advantage of his abilities.
I'll grant you that doing this in the US is going to be problematic because of ethical concerns, but there is definitely going to be some country that does it, and as soon as they do, they'll instantly gain air supremacy.
As in, we should consider why we have planes in the first place: it's to deliver some payload (bombs, missiles, or in the olden days, cameras for photography) to a specific place where you make use of that payload and then go home. Once you remove the human, you're not left with too many uses that can't be solved with existing technology.
http://www.boeing.com/history/products/himat-research-vehicl...
Rudimentary fuzzy logic in this kind of platform should be able to defeat a human pilot regardless of the pilots cunning/unpredictability/other human aspects which movies trump as superior human features. But, practical requirements such as range, payload, loiter time, pork barrelling may mean that the platform is otherwise compromised.
Additionally in an era where things such as airborne lasers are becoming a reality, the whole meta may completely change.
In practice, the losing party as often as not will try to inflict as much damage on the victor as they can, and more often than not what starts as a battle ends up being a long term occupation and that's when all simulations seem to break down.
Battles are 'easy', long term planning is not.
Heck it might even prevent war if the simulation says that both sides will suffer too high casualties to make it worth it.
Now, Germany was obviously occupied on multiple fronts.
While I can't contest that if a nation of 330 million were to devote its full human and industrial capacity for the purpose of destroying all resistance in a nation of 33 million, it would succeed...
I wouldn't call the richest country in the world failing to convert to a war economy to overcome an enemy ten times smaller then they are to be 'lack of will to win.'
The richest nation in the world should not have to convert to a war economy to defeat an enemy ten times smaller. If it could not; then it didn't want to be there in the first place, it didn't want to do what was necessary to win, or their internal politics is screwed up. Possibly more than one of those.
War is always political.
I was meaning more that a conventional war is more violence then politics. There are various aspects to war, each type has its own balance of these aspects. I just feel that in a guerilla war the politics has as much a part as violence.
It was clear then too, but it obvious now that the proto-ISIS "freedom fighter" narrative was completely divorced from reality.
At best, a very accurate simulation could tell you "in the case of war, here is the expected distribution of outcomes, here are the probabilities of various levels of "winning", here are the estimated levels of civilian casualties.
And quite a few political leaders in such situations have or would have chosen to take e.g. a 5% chance of victory instead of surrender, or simply as a deterring tactic - yes, we know that we will surely lose, but in the process you'll lose many men as well, so we'll bet that the result is not that important to you and you won't pay this price.
If Herodotus is to believed, a Persian siege against Babylon was met with the Babylonians strangling all the women except for a few to make bread as to stretch fighting resources as long as possible.
During the Sono-Soviet border conflict of the late 1960s, Mao threatened to overwhelm the Russian border with millions of Chinese as a part of his "man over weapons" strategy. Even with nuclear weapons, the Soviets were terrified of the sheer number of people that China could throw at them.
How do you predict that? How do you put a number to it?
For this to work, every aspect of the simulation would
have to match up with real world circumstances.
...which would mean no secrets.
...or weapons that, when used, obviate many degrees of
secrecy, like nuclear weapons.The Star Trek version was to have citizens of both sides executed to match simulated casualty numbers... https://en.wikipedia.org/wiki/A_Taste_of_Armageddon
Two parties agrees to wage a war in virtual worlds to decide if virtual hells should be allowed or banned. Unsurprisingly, the losing party tries to move the war in the real world to reverse the losing trend.
I don't think we'll ever see a civilized form of war where no real harm is done, because abiding by the rules of such a war is not a game-theoretic equilibrium. If a party builds a real army in addition to the simulated army and uses it, it will win the war.
Just like nuclear powers sometimes wage limited conventional wars, it's possible to imagine a set of international laws and norms where disputes would be resolved by virtual conflicts without escalation to armed force.
Advances in robotic war machines could make them so fearsome they deterred real-world armed conflict in favor of virtual conflict. (Of course, that's what they said about the machine gun before WW1)
Power for heating, cooling and hospitals. Transit systems for food distribution. Computers for synchronising all of the above.
How many casualties do you think would result from shutting down infrastructure in NYC in mid winter?
Even evacuation causes casualties. I was reading an assessment of the Fukushima evacuation that suggested fewer casualties would have resulted from just staying there.
I suspect the random variation between the two would account for far more difference than the casualties directly attributable to method of destruction, though.
Unless you mean destroying the transformers with blockbuster-size HE, or dirty nuclear devices?
I guess my point is that the number of people who would die from sustained, widespread infrastructure disruption would be huge, and not greatly changed by casualties around the blast site if you decided to blow stuff up literally instead of just metaphorically.
"Well, they're not taking [whatever the goal was] without a real fight!"
It will then be AI/humans vs AI/humans cause one side will have to bear (real) heavy losses...
war is a dirty business, but business it is, and what a juicy one. simpler people hate banks/bankers, yet I haven't heard about any protesters occupying Lockheed, BAE or similar folks and wishing them jail. last thing these powerful corporations want is to change the game that works so well for them now.
war is a seed planted for long term future. look at middle east - I guarantee you 100 years from now on, this will still be a bad place to be. who will profit from it? all that arm all sides.
when last few US wars were declared, for whatever funky official reasons, do you really believe that only US president and very few true and just in government were involved in deciding if, when, how, what tech will be used etc? in this I trust Russia much more - epicenter of power is pretty clear there.
I don't believe in some Illuminati governing everything, nor do I believe that people with real power are not corrupted to their rotten core, owning favors left and right, their missteps are overlooked and their places +-secured etc. you don't get too high in politics (or army for that matter which becomes politics from certain level) as independent and just. just my gut feeling and common sense. but boy would i be glad to be wrong!
In this same way you can figure out that conspiracies about free energy and coal/gas industry suppression of alternative energies is bullshit. Almost everyone else in the world would love to have cheaper energy so they would not allow the oil or gas companies to suppress alternative energy.
Well that's my logic anyways.
In this episode, the crew of the USS Enterprise visits a planet whose people fight a computer-simulated war against a neighboring planet. Although the war is fought via computer simulation, the citizens of each planet have to submit to real executions inside "disintegration booths" to meet the casualty counts of the simulated attacks. The crew of the Enterprise is caught in the middle and are told to submit themselves voluntarily for execution after being "killed" in an "enemy attack".
Of course, it does not always work.
Why not do hundreds of simulations, with different amounts of attacking and defending jets. Sounds like fun, must not be a problem to find pilots who want to do this simulation, it's merely hundreds of hours of gameplay :).
Or was it like, they did hundreds, but this is the only one where the AI won, and it had 4 planes while the humans had only 2?
Back in the late 1990s, Deep Blue beat the best human chess player, a demonstration of the power of AI.
Around ten years later, a tournament of individual grandmasters and individual AIs was won by ... some amateur chess players teamed with AIs.
AIs aren't good at dealing with novel situations, humans are; they complement each other (and I'll add: unlike most other endeavors, in war the environment (the enemy) is desperately striving to confuse you and do the unexpected. Your self-parking car would have more trouble if someone was trying everything they could think of to stop it, as if their survival was at stake). Also, we strongly prefer humans make life-and-death decisions; hopefully that turns out to be realistic.
If that is true, it puts this achievement in a totally different class.
Very stealthy (no bucket/glass where the pilot has to sit). Huge amounts of lift available for high G maneuvers. Infinite air-time given a re-supply tanker is in the area.
I know the train of thought. Once I spent hour thinking of "design dirt" that sticks to machine parts and protects them from real dirt and oxidation. Then it occurred to me that it's called "paint".
http://www.omicsgroup.org/journals/genetic-fuzzy-based-artif...
The concept of two flights approaching each other, seeing each other, and not engaging until they are in dogfighting range is silly. To get two modern fighters close enough for a proper turning fight at least one side will have to be taken by surprise. Otherwise, the long-range missile fight will either decide the matter or place one side in such a poor position that they will withdraw. (Either they are down or will have so reduced their energy that a turning fight isn't an option.)
This is precisely what happened when the US Navy F-16Ns were used exclusively by TOPGUN, and had a short service life before the airframe developed cracks and went to the boneyard. These jets were flown harder than the USAF flew them in operational service.
Also, the aerodynamic control surface effectiveness on fighter jets does not allow rapid reversals at some magic level significantly beyond what a human pilot can handle. An old A-4 Skyhawk can roll beyond 360 degrees a second with a human pilot.
Another issue is that thrust vectored aircraft like the F-22 Raptor, will rapidly loose airspeed at a relatively low sustained turn rate. This turn rate is way less than the 9G maximum of the aircraft. Additionally, An F-22 (91-4003) was effectively written off after a -11G overstress mishap. The pilot was okay, but the jet never flew again. To fly a fighter in an expanded G envelope would require additional weight, negating some of the benefits of unmanned fighters.
the designers are targeting a certain g-limit assumed for the pilots, and then make the plane as light as possible.
perhaps we will see hybrids in the intermediate term : humans providing guidance to the unmanned craft (with latency and jammability) and the craft autonomously doing the low latency fighting. i'm not sure this is "worse" if all the planes are unmanned. granted any fighting is bad.
this also has big implications for design -- reliability requirements, disposability change. one way missions.
the comments re: once the virtual fight ends the humans start fighting, i don't think so. they won't be as good, else the ai version wouldn't be in service, so that would be pointless.
If the plan was for smaller, miniature, one-way drones, then something like a ADM-160 MALD or AGM-154 JSOW would suffice, but it wouldn't be a fighter sized drone.
Also, do the world a favor, and don't innovate new weapons. They leave an indelible affect on the collective mind.
This is changing very rapidly with hardware-accelerated RNN chips being researched by Google and facebook.
I wonder about communication though. All the enemy fighter needs to do is jam any signals used by the jets to communicate. I wonder if they could rely on laser/line-of-sight communication instead of RF frequencies.
Now we just need the AI to go rogue and target it's master ;)
"Dark Star - Bomb Philosophy- https://www.youtube.com/watch?v=29pPZQ77cmI
and the conclusion "Dark Star - let there be light"- https://www.youtube.com/watch?v=I9-Niv2Xh7w
I feel like swarms of cheap, fast, unmanned aircraft that communicate efficiently and orchestrate their behavior with AI will prevail in the end.
Are there sensors that are immune to scrambling and bad data?
"The AI, known as Alpha, used four virtual jets to successfully defend a coastline against two attacking aircraft - and did not suffer any losses."
"Alpha, which was developed by a US team, also triumphed in simulation against a retired human fighter pilot."
Key words here are "also" and "simulation" and "retired".
Click bait much?
https://www.youtube.com/watch?v=5hJepWBUqZk#t=0m20s
Perhaps honor can't be programmed.