When a human receives a phrase, be it a sentence or paragraph. The brain breaks it down into manageable sized phrases and clauses. Much like the predicate parser. These phrases and clauses are then further broken down into component parts. These components may be summarised as;
| Question/Order | (The nature of the phrase) |
| Object | (What the phrase refers to) |
| Subject | (What the phrase says about the object) |
| Qualifiers Size Colour | (Descriptions qualifying the object and subject) |
| Number | (All, one, two etc) |
| Location | (Where is the object) |
| Action | (Verbs) |
| Person | (You, me, us, them, they, him, her) |
| Tense | (Past, present, future or indeterminable) |
For example;
"What is the matter with Mary Jane?"
Breaks down into phrases and clauses thus;
| Subject | [What] |
| Predicate | [is the matter with Mary Jane?] |
| Object | [is the matter] |
| Phrase | [with Mary Jane?] |
Taking the phrase as a whole it breaks down into these components;
The question is WHAT IS
The order is
The object is MATTER
The qualifiers are
The colours are
The sizes are
The subject is WITH MARY JANE
The person is
The number is
The action is
The location is
The tense is present
Similarly, "It's lovely rice pudding for dinner again!" Produces;
| Subject [It] | Predicate [is lovely rice pudding for dinner again!] |
The predicate breaks down into;
Object [is rice pudding]
Description [lovely]
And produces a sub-phrase;
Phrase [for dinner again!]
Analysing the phrase as a whole we get;
The question is
The order is
The object is
The qualifiers are LOVELY
The colours are
The sizes are
The subject is LOVELY RICE PUDDING FOR DINNER AGAIN
The person is
The number is
The action is
The location is
The tense is indeterminable
"This is the house that Jack built" produces the following analysis. Compare it with the output of the syntactical parser described in a previous chapter.
Subject [This]
Predicate [is the house that Jack built]
The predicate breaks down into an object and a sub-phrase;
Object [is the house]
Phrase [that jack built]
The sub-phrase breaks down into an object and an action (verb);
Action [built]
Object [that Jack]
Analysing the phrase as a whole we get the following summary;
The question is
The order is
The object is HOUSE
The object qualifiers are
The object colours are
The object sizes are
The subject is JACK
The person is
The number is
The action is BUILT
The location is
The tense is indeterminable
A component parser is a fairly complex affair. But can be a useful addition to existing parsers. The component parser has some similarities to the predicate parser. The lexicon of verbs (actions) is the same for example.
--------- Listing 5 --------- /* COMPONENT PARSER */ /*---------------*/ /* Include files */ /*---------------*/ #includeIn trying to understand a phrase, extensive use of the component parts is made, with occasional referrals back to the phrase as a whole. The problems of "understanding", rather than "responding" are discussed in a later chapter.#include #include #include /*-----------*/ /* Constants */ /*-----------*/ #define num_a 140 #define num_p 12 #define num_q 41 #define num_l 20 #define num_n 37 #define num_c 14 #define num_f 17 #define num_qs 19 #define num_s 9 #define num_o 3 /*-----------*/ /* Lexicons */ /*-----------*/ static char *actions[num_a] = { "LIKE" , "DISLIKE", "LOVE" , "HATE" , "SLEEP" , "PUMP", "MAKE" , "WEAR" ,"RENOUNCE", "SHORTEN", "KEEP", "FEED", "DESCEND" , "CONVERT", "KEEP", "OPEN" , "EAT" , "READ" , "DRY" , "KILL" , "MEND","CAUSE", "SIT" , "SAT" , "CLOSE", "TAKE", "RUN" , "JUMP", "SKIP", "HOP", "BREAK", "INVENT", "DISCOVER", "EXPLORE", "PUNCH","KICK", "IGNITE", "CHAT", "WRITE", "TALK","LISTEN", "WHISTLE","SHOUT","SCREAM","DRINK","BATHE", "WASH","CROSS", "ENTER", "CLIMB", "TIE", "BREATHE", "SNIFF", "SPIT" ,"GO","PUT","DRINK", "LOOK", "TURN", "BORN","LEAVE", "URINATE", "SMOKE", "LIGHT", "DESIGN" , "DRAW" ,"PAINT", "BOIL", "STRAIN", "POUR","TELEPHONE","PHONE", "CALL" ,"DRESS", "CUT", "SAW", "GONE","STUDY", "FOLD" , "PLAY", "PAY", "EARN","MEET", "MET", "LIVE","DWELL", "BEEN", "HEAR" , "HEARD", "SING" , "SUNG", "DEVELOP", "STIR","SEE", "SEEN", "RING", "FEEL" , "FELT", "DIED", "HUNT", "ASK", "STAY" ,"KISS", "CUDDLE", "GIVE","BUY", "BOUGHT", "LICK" , "TASTE", "ADMIRE" ,"PROVIDE", "COME", "BUILD", "BUILT", "REFERENCE" ,"WENT", "CONSTRUCT","LEAN","WALK","CARRY","RIDE", "ORBIT","DRIVE" , "DROVE" , "DOWN", "ALONG", "SHALL","SHOULD","THINK", "THOUGHT","GOT", "CROUCH","CRAWL", "GET","LEND", "LENT","HAVE", "HAS" , "HAD","SUGGEST" }; static char *questions[num_qs] = { "WHAT", "WHEN", "HOW", "WHY", "WHERE", "WHO","DO", "DOES" ,"ARE", "IS", "WAS", "CAN" ,"AM" ,"WHICH","HAVE" , "WILL" , "DID" , "HAS", "SHOULD"}; static char *orders[num_o] = { "USE" , "TELL", "QUIT" }; static char *fillers[num_f] = { "IS", "THE" , "A", "AN" , "ARE" ,"TOO","ALSO", "OF" , "COME" , "FROM" , "ITS" , "THEIR" , "THAT" , "ABOUT", "THING" , "THIS" , "BECAUSE" }; static char *qualifiers[num_q] = {"FAST", "SLOW", "ROUND", "SQUARE", "HAIRY", "BALD", "FRIGHTENING","INTERESTED", "SHY", "NICE", "HORRIBLE" , "LOVELY","INTERESTING", "PRETTY" , "BEAUTIFUL" , "BLONDE" , "BRUNETTE" , "SEXY" , "SHINY" , "DULL" , "BORING" , "CAREFUL" , "HARD" , "SOFT" , "GENTLE" , "QUICK" , "LIGHT", "HEAVY", "RADIOACTIVE", "METALLIC","TIRED", "GASEOUS", "BRIGHT", "SWEET", "SOUR" , "BITTER", "GOOD","BAD","DRUNK","STUPID","MUTUAL" }; static char *sizes[num_s] = {"BIG", "SMALL", "FAT", "THIN", "TALL" ,"SHORT", "LARGE", "PETITE", "TINY" }; static char *colours[num_c] = { "BLACK", "BLUE", "RED", "YELLOW", "ORANGE", "GREEN","BROWN" , "GINGER" , "PURPLE", "MAUVE", "SILVERY", "SILVER", "GOLD", "CYAN" }; static char *numbers[num_n] = { "ALL" , "SOME" , "FEW" , "MANY" , "MOST" , "COUPLE","NEXT","PREVIOUS","ONE", "TWO", "THREE" , "FOUR", "EVERY", "LAST", "FIVE", "SIX", "SEVEN","EIGHT", "NINE", "TEN", "ELEVEN","TWELVE", "THIRTEEN","FOURTEEN", "FIFTEEN", "SIXTEEN", "SEVENTEEN", "EIGHTEEN", "NINETEEN","TWENTY", "THIRTY", "FORTY", "FIFTY", "SIXTY","SEVENTY", "EIGHTY", "NINETY" }; static char *pronouns[num_p] = { "THEM", "THEY", "WE", "HIM", "HER" ,"US" , "HE", "I", "YOU" ,"MY", "YOUR" , "ME"}; static char *locations[num_l] = { "IN" , "ON", "BESIDE" , "EAST" , "WEST" , "NORTH" ,"SOUTH" , "UNDER" , "UP" , "DOWN", "BELOW", "ABOVE", "RIGHT", "LEFT", "OVER","INTO","AROUND" , "TO", "OFF", "THERE" }; /*------------*/ /* Work areas */ /*------------*/ static char last_subject[80]; static char last_object[80]; /*---------------------*/ /* Function prototypes */ /*---------------------*/ int IS_COLOUR(char *); int IS_FILLER(char *); int IS_LOCATION(char *); int IS_NUMBER(char *); int IS_PRONOUN(char *); int IS_SIZE(char *); void strchg(char *, char *, char *); void truncstr(char *,int ); /*--------------------*/ /* Start of functions */ /*--------------------*/ static void GET_QUESTION(char *text, char *question) { /* Extract a question from the phrase string */ char *p; char first_word[40]; char save; int n; int more; strcpy(question,""); do { more = 0; p = strpbrk(text," ,.;:?!"); save = *p; *p = 0; strcpy(first_word,text); *p = save; for(n = 0; n < num_qs; n++) { if (stricmp(first_word,questions[n]) == 0) { /* store question */ if (*question) strcat(question," "); strcat(question,first_word); /* remove question from text */ if (p) { while(strchr(" ,.:;?!",*p) && *p) p++; if (*p) memmove(text,p,strlen(p) + 1); } else *text = 0; more = 1; } } } while(more); } static void GET_ORDER(char *text, char *order) { /* Extract an order from the phrase string */ char *p; char first_word[40]; char save; int n; strcpy(order,""); p = strpbrk(text," ,.;:?!"); save = *p; *p = 0; strcpy(first_word,text); *p = save; for(n = 0; n < num_o; n++) { if (stricmp(first_word,orders[n]) == 0) { /* store order */ strcpy(order,first_word); /* remove order from text */ while(strchr(" ,.:;?!",*p)) p++; memmove(text,p,strlen(p) + 1); return; } } } static void GET_OBJECT(char *text, char *object, char *action) { /* Obtain object from phrase string */ char *p; int n; int ignore; char temp[30]; char copy[250]; strcpy(copy,text); p = strtok(text," ,.;:?!"); while(p) { ignore = 0; if (stricmp(p,action) == 0) ignore = 1; if (!ignore) ignore = IS_FILLER(p); if (!ignore) ignore = IS_NUMBER(p); if (!ignore) ignore = IS_COLOUR(p); if (!ignore) ignore = IS_SIZE(p); if (!ignore) ignore = IS_LOCATION(p); if (!ignore) ignore = IS_PRONOUN(p); if (!ignore) { for (n = 0; n < num_q; n++) { if (stricmp(p,qualifiers[n]) == 0) ignore = 1; else { strcpy(temp,qualifiers[n]); if (temp[strlen(temp) - 1] == 'E') { temp[strlen(temp) - 1] = 0; if (temp[strlen(temp) - 1] == 'L') temp[strlen(temp) - 1] = 0; } strcat(temp,"LY"); if (stricmp(p,temp) == 0) { ignore = 1; break; } } } } if (!ignore) { for(n = 0; n < num_qs; n++) { if (stricmp(p,questions[n]) == 0) { ignore = 1; break; } } } if (!ignore) { if (stricmp(p,"IT") == 0) strcpy(object,last_object); else strcpy(object,p); strcpy(text,copy); p = strstr(text,object); strcpy(last_object,""); if (p) { /* Capture preceding preposition */ p--; while(*p == 32) p--; while(*p != 32 && p >= text) p--; if (*p == 32) p++; if (strncmp("THE ",p,4) == 0) strcpy(last_object,"THE "); else if (strncmp("the ",p,4) == 0) strcpy(last_object,"the "); } strcat(last_object,object); return; } p = strtok(NULL," ,.;:?!"); } strcpy(text,copy); strcpy(object,""); } static void GET_SUBJECT(char *text, char *subject, char *object, char *action) { /* Obtain subject from phrase string */ char *p; int ignore; char copy[250]; strcpy(copy,text); strcpy(last_subject,subject); strcpy(subject,""); /* Restrict phrase to that part following the object */ p = strstr(text,object); if (p == NULL) p = strstr(text,"IT "); memmove(text,p,strlen(p) + 1); p = strtok(text," ,.;:?!"); while(p) { ignore = 0; if (stricmp(p,object) == 0) ignore = 1; if (!ignore) ignore = IS_FILLER(p); if (!ignore) ignore = IS_LOCATION(p); if (!ignore) ignore = IS_PRONOUN(p); if (!ignore) ignore = IS_COLOUR(p); if (!ignore) ignore = IS_SIZE(p); if (!ignore) ignore = IS_NUMBER(p); if (stricmp(p,action) == 0) ignore = 1; if (!ignore) { if (stricmp(p,"IT")) { if (*subject) strcat(subject," "); strcat(subject,p); } } p = strtok(NULL," ,.;:?!"); } strcpy(text,copy); } static void GET_OBJECT_QUALIFY(char *text, char *object_qualify, char *subject) { /* Obtain qualifying statements from phrase string */ char *p; int n; int ignore; char oq[78]; char copy[250]; char temp[30]; strcpy(copy,text); strcpy(object_qualify,""); p = strtok(text," ,.;:?!"); while(p) { for(n = 0; n < num_q; n++) { if (stricmp(p,qualifiers[n]) == 0) { if (*object_qualify) strcat(object_qualify,","); strcat(object_qualify,p); break; } else { strcpy(temp,qualifiers[n]); if (temp[strlen(temp) - 1] == 'E') { temp[strlen(temp) - 1] = 0; if (temp[strlen(temp) - 1] == 'L') temp[strlen(temp) - 1] = 0; } strcat(temp,"LY"); if (stricmp(p,temp) == 0) { if (*object_qualify) strcat(object_qualify,","); strcat(object_qualify,p); break; } } } p = strtok(NULL," ,.;:?!"); } strcpy(text,copy); ignore = 0; strcpy(oq,object_qualify); p = strtok(object_qualify,","); while(p) { if (stricmp(p,subject) == 0) { ignore = 1; break; } p = strtok(NULL,","); } strcpy(object_qualify,oq); if (!ignore) for (n = 0; n < num_q; n++) { if (stricmp(subject,qualifiers[n]) == 0) { if (*object_qualify) strcat(object_qualify,","); strcat(object_qualify,subject); strcpy(subject,""); break; } } } static void GET_NUMBER(char *text, char *number) { /* Obtain number words from phrase string */ char *p; char copy[250]; int n; strcpy(copy,text); strcpy(number,""); p = strtok(text," ,.;:?!"); while(p) { for(n = 0; n < num_n; n++) { if (stricmp(p,numbers[n]) == 0) { if (*number) strcat(number,","); strcat(number,p); break; } } p = strtok(NULL," ,.;:?!"); } strcpy(text,copy); } static void GET_WHO(char *text, char *who) { /* Obtain first or second person references from phrase string */ char *p; char copy[250]; int n; strcpy(copy,text); strcpy(who,""); p = strtok(text," ,.;:?!"); while(p) { for(n = 0; n < num_p; n++) { if (stricmp(p,pronouns[n]) == 0) { if (*who) strcat(who,","); strcat(who,p); break; } } p = strtok(NULL," ,.;:?!"); } strcpy(text,copy); } static void GET_COLOUR(char *text, char *colour) { /* Obtain colour words from the phrase string */ char *p; char copy[250]; int n; strcpy(copy,text); strcpy(colour,""); p = strtok(text," ,.;:?!"); while(p) { for(n = 0; n < num_c; n++) { if (stricmp(p,colours[n]) == 0) { if (*colour) strcat(colour,","); strcat(colour,p); break; } } p = strtok(NULL," ,.;:?!"); } strcpy(text,copy); } static void GET_SIZE(char *text, char *size) { /* Obtain size description words from the phrase string */ char *p; char copy[250]; int n; strcpy(copy,text); strcpy(size,""); p = strtok(text," ,.;:?!"); while(p) { for(n = 0; n < num_s; n++) { if (stricmp(p,sizes[n]) == 0) { if (*size) strcat(size,","); strcat(size,p); break; } } p = strtok(NULL," ,.;:?!"); } strcpy(text,copy); } static void GET_LOCATION(char *text, char *location) { /* Obtain location description words from the phrase string */ char *p; char copy[250]; int n; strcpy(copy,text); strcpy(location,""); p = strtok(text," ,.;:?!"); while(p) { for(n = 0; n < num_l; n++) { if (stricmp(p,locations[n]) == 0) { if (*location) strcat(location,","); strcat(location,p); break; } } p = strtok(NULL," ,.;:?!"); } strcpy(text,copy); } static void GET_ACTION(char *text , char *action) { /* Obtain action words from the phrase string */ char *p; char n; char word[50]; char pre[50]; char copy[250]; strcpy(copy,text); strcpy(action,""); strcpy(pre,""); p = strtok(text," ,.;:?!"); while(p) { for(n = 0; n < num_a; n++) { strcpy(word,actions[n]); if (stricmp(p,word) == 0) { if (stricmp(pre,"THE")) { if (*action) strcat(action,","); strcat(action,p); } break; } else { /* Past tense */ if (word[strlen(word) - 1] == 'E') strcat(word,"D"); else if (word[strlen(word) - 1] == 'Y') { /* Irregular words! */ if (stricmp(word,"PAY") == 0) strcpy(word,"PAID"); else if (stricmp(word,"PLAY") == 0) strcpy(word,"PLAYED"); else { word[strlen(word) - 1] = 0; strcat(word,"IED"); } } else strcat(word,"ED"); if (stricmp(p,word) == 0) { if (*action) strcat(action,","); strcat(action,p); break; } else { /* Plural */ strcpy(word,actions[n]); strcat(word,"S"); if (stricmp(p,word) == 0) { if (stricmp(pre,"THE")) { if (*action) strcat(action,","); strcat(action,p); break; } } else { /* Active */ strcpy(word,actions[n]); strcat(word,"ING"); if (stricmp(p,word) == 0) { if (stricmp(pre,"THE")) { if (*action) strcat(action,","); strcat(action,p); break; } } } } } } strcpy(pre,p); p = strtok(NULL," ,.;:?!"); } strcpy(text,copy); } static int IS_FILLER(char *p) { /* Return 1 if the word is a filler word */ char n; for(n = 0; n < num_f; n++) { if (stricmp(p,fillers[n]) == 0) return(1); } return(0); } static int IS_NUMBER(char *p) { /* Return 1 if the word is a number */ int n; for(n = 0; n < num_n; n++) { if (stricmp(p,numbers[n]) == 0) return(1); } return(0); } static int IS_COLOUR(char *p) { /* Return 1 if the word is a colour */ char n; for(n = 0; n < num_c; n++) { if (stricmp(p,colours[n]) == 0) return(1); } return(0); } static int IS_SIZE(char *p) { /* Return 1 if the word is a size description word */ char n; for(n = 0; n < num_s; n++) { if (stricmp(p,sizes[n]) == 0) return(1); } return(0); } static int IS_LOCATION(char *p) { /* Return 1 if the word is a location description */ char n; for(n = 0; n < num_l; n++) { if (stricmp(p,locations[n]) == 0) return(1); } return(0); } static int IS_PRONOUN(char *p) { /* Return 1 if the word is a first or second person reference */ char n; for(n = 0; n < num_p; n++) { if (stricmp(p,pronouns[n]) == 0) return(1); } return(0); } void NLP(char *text, char *question, char *order, char *object, char *object_qualify, char *subject, char *number, char *action, char *location, char *colour, char *size, char *who, char *tense) { /* Parses a simple phrase and returns the component parts */ char *p; char *q; /* DON'T must be protected from expansion */ strchg(text,"DON'T","DON|T"); /* Expand "'RE" */ strchg(text,"'RE"," ARE"); /* Expand "'S" */ strchg(text,"'S"," IS"); /* Expand I'M */ strchg(text,"I'M","I AM"); /* Expand "N'T" */ strchg(text,"N'T"," NOT"); /* Expand "I'VE" */ strchg(text,"I'VE","I HAVE"); /* Expand "'re" */ strchg(text,"'re"," are"); /* Expand "'s" */ strchg(text,"'s"," is"); /* Expand I'm */ strchg(text,"I'm","I am"); /* Expand "n't" */ strchg(text,"n't"," not"); /* Expand "I've" */ strchg(text,"I've","I have"); /* DON'T must be protected from expansion */ strchg(text,"DON|T","DON'T"); GET_QUESTION(text, question); GET_ACTION(text , action); GET_ORDER(text,order); GET_OBJECT(text,object,action); GET_SUBJECT(text, subject, object, action); GET_WHO(text,who); GET_OBJECT_QUALIFY(text, object_qualify, subject); GET_COLOUR(text,colour); GET_SIZE(text,size); GET_NUMBER(text , number); GET_LOCATION(text,location); printf("\nThe question is %s",question); printf("\nThe order is %s",order); printf("\nThe object is %s",object); printf("\nThe object qualifiers are %s",object_qualify); printf("\nThe object colours are %s",colour); printf("\nThe object sizes are %s",size); printf("\nThe subject is %s",subject); printf("\nThe person is %s",who); printf("\nThe number is %s",number); printf("\nThe action is %s",action); printf("\nThe location is %s",location); printf("\nThe tense is "); if (strstr(question,"WAS")) *tense = 1; else if (strstr(question,"IS")) *tense = 2; else if (strstr(question,"DO")) *tense = 2; else if (strstr(question,"IS")) *tense = 2; else if (strstr(question,"DID")) *tense = 1; else if (strstr(text,"GOT ")) *tense = 2; else if (strstr(question,"WILL")) *tense = 3; else *tense = 0; if (*tense == 0) { /* Perhaps action gives a clue */ if (*action) { if (strchr(action,',') == NULL) { p = &action[strlen(action) - 2]; if (stricmp(p,"ED") == 0) *tense = 1; } else { q = action; do { p = strchr(q,','); q = p - 1; if (*q == 'S') { *tense = 2; break; } q--; if (stricmp(q,"ED") == 0) { *tense = 1; break; } q = p + 1; } while(p); } } } switch(*tense) { case 1: printf("past"); break; case 2: printf("present"); break; case 3: printf("future"); break; default: printf("indeterminable"); } } void main() { char a[80]; /* question */ char b[80]; /* order */ char c[80]; /* object/directive */ char d[80]; /* object qualifier */ char e[80]; /* subject */ char f[80]; /* number */ char g[80]; /* action */ char h[80]; /* location/direction */ char i[80]; /* Colour */ char j[80]; /* size */ char k[80]; /* who */ char tense; /* Tense, past present or future */ char text[100]; do { printf("\nEnter a phrase\n"); gets(text); if (*text != 0) { strupr(text); NLP(text,a,b,c,d,e,f,g,h,i,j,k,&tense); } } while(*text); }
By breaking a phrase down into components understanding various forms of the same phrase becomes easier. Let's take the basic phrase, "I'M BORED". This can have variations such as "I AM BORED", "I AM VERY BORED", "I AM EXTREMELY BORED" and so on. But the basic message is the same, "I AM BORED". Without breaking the phrase into components we have to compare the phrase against a vast lexicon of known phrases. However, with components we only need to compare the object and subject, either of which will be set to "BORED" in all the previous examples of the same basic message.
The component parser decides whether a phrase is a question, or not by extracting the first word of the phrase and comparing it against a lexicon of known questions; WHO, WHAT, WHERE, WHEN, WHY, WOULD, SHOULD, DO, DID, IS, ARE, WILL, CAN. If a match is found then the phrase is a question. For example;
"Where is the cat?"
The first word is extracted, "WHERE", and upon comparing it against the lexicon of questions a match is found. So this phrase is a question.
"Do you think Arsenal will win the match?"
The first word is extracted, "DO", and upon comparing it against the lexicon of questions a match is found. So this phrase is a question.
You might recognise this technique from the slot-and-frame parsers described earlier.
A further stage of component parsing is deciding the nature of a question or statement. questions are of two different types; request and interrogation. Although both request something, the interrogation requests further information. Human's tend to communicate ambiguously, as in the question; "Have you got a light?" Which, under some circumstances actually means; "Can you give me a source of ignition". Under other circumstances it means, "Can I have a source of light". Finally this phrase may actually be an interrogation asking if you have a source of light.