I agree, if you're going to automate debate, you really need to have your software comprehend what an argument is. I have a design in my head of such a system that I've been wanting to develop for years but have never put it at the front of my priorities. Last time I was pondering implementing it, I looked around online and was disappointed to see that there were about 5 attempts to do structured arguing as a web service, but they were all FAR too informal and unstructured. I don't want a place to chat and type out paragraphs of pros. I want some hardcore step-by-step argumentation! Might as well talk about it, though, as it's interesting.
I'd go back to basics. Take Logic by W C Salmon (http://www.ditext.com/salmon/logic.html) and treat is a user spec. What you want, I think, is a way to structure arguments and catalog them like a wiki. So you have these things that people can interact with:
1. A premise
2. A conclusion
3. An argument
If I'll be allowed to model the data with Haskell data types to give a more formal description:
data Claim = Claim Text
data Premise = Axiom Claim | Conclusion Argument
data Argument = Argument [Premise] Claim
In more verbose English: an Argument is a list of Premises and a final conclusion which is some plain-text Claim. Claims are either facts or axioms that are taken as read. A Premise is either an Axiomatic claim or a Conclusion from another argument.
Only one conclusion can be drawn in an argument at once. If you want to make compound arguments that draw multiple conclusions, that's where the duplicity of a Premise comes in. It is either a plain-text claim, or a reference to another argument's conclusion that has been made earlier.
So putting that into practice:
humans_mortal_claim = Claim "All humans are mortal."
men_humans_claim = Claim "All men are humans."
men_mortality_argument =
Argument [humans_mortal_claim
,men_humans_claim]
(Claim "All men are mortal.")
Given those claims and argument I can construct a new argument:
plato_manhood_claim = Claim "Plato is a man."
plato_mortality_argument =
Argument [plato_manhood_claim
,men_mortality_argument]
(Claim "Plato is mortal.")
It's very simple, but it models the argument perfectly -- BUT, with the flexibility of simple English. You don't have to learn first-order logic here or try to parse the English into some formal representation (although I am currently working on such a library for Haskell: http://chrisdone.com/posts/attempto-controlled-english).
Now, that's the data model. The UI would be to represent arguments as squares on the screen consisting of a list of premises that you can add to and a final conclusion. When you add a premise, you can either pick from a catalogue of existing axioms/facts, or choose from existing conclusions of other arguments. The conclusion is a plain text field where you make an original, new claim, derived from the premises.
But how do you make use of it and deal with real-world contention? So, when someone proposes an argument, I have a set of UI features to help me accept or refuse it. First, I have to go through each of the premises one by one and accept them. If they refer to conclusions of arguments I've already reviewed and accepted before, then they will be pre-accepted. Finally, I decide at the end whether THE PREMISES SUPPORT THE CONCLUSION, and NOT, whether I agree with the conclusion as-is (e.g. I can argue that the moon is made of cheese with valid argumentation, even though it is known to be false).
So you decide that, indeed, the premises support the conclusion, so you mark the argument as valid. HOWEVER, the argument is ONLY accepted if you marked all the premises as accepted. Because, as Salmon clearly states, an argument's conclusion is only considered true if all the premises that support it are true. Indeed, the opposite case may be true: you accept the conclusion, but the argument is invalid. This is the difference between argumentative validity and existential truth.
I believe dealing with both, and being explicit about it, is very handy. In one case pointing them out in the negative is helpful to avoid error. In the other case, saying "I agree with the conclusion even though the argument is invalid to me" or "I don't agree with the conclusion/one of the premises, but the argument itself is valid" is a good way to give encouragement to opponents, that you're not wholesale saying "you're just WRONG". You're saying "this part is not quite acceptable to me, but this part is okay, we just need to work on this bit before I can accept it."
So it may be and should regularly be the case that e.g.
1. I accept all the premises in this argument, but the argument is invalid or incomplete.
2. I accept none or only some of the premises, but the argument itself is valid.
3. I accept none of the premises, but the conclusion I accept.
Now, let's consider the cases of decisions made by a reviewer:
1. Argument is valid, conclusion accepted: this is perfect. This argument can now be considered part of the accepting user(s) knowledge-base.
2. Argument is valid, one or more premises not accepted: more work is needed. The user needs to contest one of the premises and start a new, separate argument, but linked to this premise, to address/resolve it. Some first-class support could be available, like a button to say "Please cite this premise with a source". And the user would get a notification saying "someone wants a citation", or whatever.
3. Argument is invalid: the user needs to start a separate argument to resolve it. One nice feature of the UI could be to select from a list of formal fallacies which the argument exhibits, each of which would have links describing what the fallacy entails. This would help users arguing sincerely to overcome fallacies.
4. Argument is valid, premises accepted, conclusion not accepted: the user has made a mistake in assessing the premises and the argument. You can't have an argument in which all the (accepted) premises support the conclusion, and yet disagree with the conclusion, if you're being sincere. There could be some helpful links about argument structure here.
People would just submit arguments and axiomatic claims into the ether, and other people can come along and accept them or reject them as part of their own personal knowledge-base. I think you would have to distinguish between public and personal "knowledge". It would be like Github, you fork people's arguments/claims by accepting them.
I'm not sure that this would be as awesome as I think it would be, because it'd require a great deal of sincerity. Just thinking about using it rules out a lot of contentious debates which are in essence people expressing their feelings rather than trying to reach the truth. However, it would certainly be an efficient way to nail down precisely where you disagree with someone, because you'd be able to point it out, in the UI. This here, this precise premise I do not accept. And either you both could split it off and argue that premise or just leave cancel the whole debate. Ultimately, it could be a nice knowledge-base. And a way for people who normally wouldn't know the first thing about arguing effectively to, with the AID of the system, develop an intuition for the structure of a decent, honest argument. I'd really try to downplay any "voting" aspect that gives credence to popularity. That would be my dream.
Sorry for spamming your face off. I'll keep this comment for later if I eventually go ahead and implement this.