Home | Videos | Contact Us   




                         << content                                      Chapter 8

Decision Making and Planning Tasks

There are many different levels on decision making and each level influences the way the robot makes decisions. Below is an outline of the level of factors that will influence the robot (the AI program) in terms of decision making.

Levels of Decision Making:  

1. innate reflexes based on pain--when a person is in great pain reflexes are most likely to trigger. These reflexes are wired into pain so that when the pain reaches a certain point it triggers the reflex. No conscious decision was needed to trigger this action. Some of these innate reflexes are: when a person is in great pain he/she yells out loud, when the knee cap is hit with a hammer the leg moves automatically, etc.

2. Learned decisions based on past knowledge--the conscious guides the robot to make decisions. These decisions are either based on future predictions or logical decision making.

3. Pain and pleasure built into the robot--attractiveness or ugliness, physical pain (degree of pain) and physical pleasure (degree of pleasure) are factors that the robot uses to make decisions. Is the robot going to eat lobster for dinner (the robot loves lobsters) or is the robot going to eat rice? (the robot eats rice only if he has to). These pain/pleasure factors that are built into the robot will make decisions.

4. Daily routine--Learned things that the robot was thought everyday by teachers are factors in decision making. Some of these daily routines are so natural that the robot doesn't need to make a decision to do them. Daily routines such as: waking up in the morning, brushing your teeth, eating 3 meals a day based on the time, going to sleep at night, using the bathroom when you need to go, and going to work or school on weekdays.

These are the levels of decision making. The higher levels overshadow the lower levels in terms of decision making. For example, innate pain overshadows learned decisions because innate pain is a reflex and is triggered by pain while learned decisions uses conscious thoughts to make decision. In other words, innate pain is triggered without conscious thought and overshadows learned decisions.

Another example is learned decisions can overshadow pain and pleasure. This form of pain and pleasure doesn't trigger reflexes, it's just a lower degree of pain/pleasure from innate reflexes--a degree where the robot can manage the pain. If a person has an itch on his butt and this person is walking on the street, the person can make a decision to scratch his butt or not. He can wait until he gets to a private area before scratching his butt. This is one demonstration of learned decision having higher priority than pain/pleasure. Even something like using the bathroom require learned decisions. If you have to go the pain is unbearable. However, you can't take a dump on the street or in a classroom. You have to make a decision to go to the bathroom and take a dump. Even though the pain is so great the learned decisions guided the robot to take the appropriate actions.

Pain and pleasure is another factor that is used for hidden objects. The AI finds these patterns and wire pathways with pain and pleasure. The strongest pathways have their powerpts strengthened because it's wired to pleasure and the weak pathways have their powerpts lowered because it's wired to pain. The learned decision encapsulates pain/pleasure to plan out tasks and make decisions for the robot. The main function of decision making is always to pursue pathways that lead to pleasure and stay away from pathways that lead to pain.

Daily routines such as brushing your teeth, sleeping at 9 pm, and waking up at 7 am are just things that we learn everyday and this type of learning is so normal that we do them without thinking. Learned decision can overshadow these things because we can control when we sleep by conscious thought. Instead of 9 pm we can sleep at 2 am. Instead of eating cereal for breakfast we can eat a hamburger. This daily routine is also another factor that can be encapsulated into learned decisions to plan out tasks and make decisions for the robot. The AI program finds patterns concerning daily routines and use this pattern in a hidden object. This hidden object will then be assigned to words/sentences as meaning of words/sentences.

In this chapter we will concentrate on learned decisions.  Learned decisions are lessons that are taught to the robot by teachers.  The teachers teach the robot how to make decisions.  Many decision making lessons will be taught to the robot.  When the robot is in the supermarket to buy food he has to decide what foods to buy.  When the robot has free time, he has to decide what he wants to do.  When the robot has extra money in the bank, he has to decide what to buy and for whom. 

Thus, for each situation the decision making process is different.  In order to create a universal pathway to make a decision for the current environment the robot must be taught decision making under many situations.


Computer programs inside the conscious

The robot will sense data from the environment through its 5 senses:  sight, sound, taste, touch and smell.  Target objects are objects the robot recognizes from the environment.  The brain of the robot will extract element objects that have strong association with their respective target objects.  All element objects from all target objects will compete with one another to be activated in the mind.  These activated thoughts are known as the conscious of the robot. 

There are two types of conscious thoughts:  open activation and hidden activation.  The open activations are element objects that are presented to the conscious and the robot is aware of the data.  On the other hand, hidden activations are element objects that are not presented to the conscious and the robot is unaware of the data.  Extremely complex tasks in memory, such as problem solving, will require both open and hidden activation. 

Think of the conscious as a “memory bubble” that takes in and modify data; and the modified data instruct the robot to take action (FIG. 42A - 42B).  Inside the conscious is a computer program (or a series of interconnected computer programs) that manipulate data in the conscious.  The intelligent pathways extracted from memory and used by the robot generate this computer program.

 FIG. 42A




The intelligent pathways control the data in the conscious by adding, deleting and manipulating data.  They also control the searching and extracting of data from memory.  The instructions inside the intelligent pathways will have search functions that will extract relevant data, based on a situation, and put them into the conscious.  Some data will be extracted from memory, while other data will be extracted from long-term memory. 

There are also other things the intelligent pathways in memory can do.  They can control the comparing of data; or analyze two or more data from the conscious.  These intelligent pathways can even predict the future and generate a computer program to output future events of what might happen in the future based on the current environment. 

The conscious actually does many other things for the robot.  Some of these things include:  giving knowledge about an object, providing meaning to language, solving problems, answering questions, following orders, planning tasks, solving interruptions of tasks, managing multiple tasks at that same time, following the rules of a game, coming up with imagination and so forth.  The intelligent pathways in memory allow the conscious of the robot to do all these things (and more). 


Important data in the conscious

The conscious does many different things for the robot.  Four of the most important things the conscious does are:  1.  manage tasks.  2.  establish rules to follow, based on the tasks.  3.  solving problems and planning steps to achieve tasks.  4.  know identity.  Referring to FIG. 42B, there are four containers the computer program in the conscious generated as a result of intelligent pathways:  the task container, the rules container, the planning container and the identity container.  All data from all four containers influence each other one way or another.  For example, the rules will influence what tasks to follow/abort and the planning information will influence what rules to follow or what tasks to do. 

These containers are just temporary caches inside the conscious that was generated by intelligent pathways in memory.  Based on the current environment, the robot selects an optimal pathway from memory and that optimal pathway has instructions to create containers so that groups of data could be manipulated and logical thoughts and actions can be had by the robot.  The intelligent pathways create any type of computer program or discrete mathematical functions to manipulate data in the conscious -- a database system, an operating system to manage multiple threads, a word processor, an image processor, a search engine, a state machine or any software program. 


Managing multiple tasks

The robot’s brain has a task container that manages tasks.  When he wakes up in the morning, knowledge will pour into the robot’s conscious in terms of 5 sense data.  Most likely sentences will pour into the task container.  Natural tasks that the robot does everyday will be the first things to populate the container.  Natural tasks will include:  brush your teeth in the morning, eat breakfast, eat 3 meals a day, take a shower at night, go to work at 8:30 am, leave work at 5:00 pm and so forth. 

Previous tasks that the robot decided to do will also be included in the task container such as decisions made yesterday or last week or last year.  For example, if the robot decided 2 weeks ago that he was going on vacation next year December, that task might be included in the task container.  If the robot decided yesterday that he was going to eat hamburger for breakfast the very next day (which is today), then the task of eating hamburger for breakfast will be in the task container. 

Tasks in the task container can be ongoing as well, if the robot was a student in school, then his primary task in the task container will be to go to school at 7:30 am every morning.  If the robot was a business man, then his primary task in the task container will be to go to work at 8:30 am every morning.  The tasks in the task container are not fixed and are changeable.  For example, if the robot graduates from school and found a job at a local business, then every day that he wakes up the task of going to work at 8:30 am will be there.  

Most of the task in the task container is “at the moment” and the robot does a task, then move on to other tasks and so forth.  Future tasks are remembered in memory and it’s the job of the robot’s conscious to remind the robot what tasks to do at certain times.  This robot forgets things so sometimes, he forget to do tasks.  Sentence commands can also define a task to do at certain times or certain places.  For example, if a friend says, “call me when you get home from work”, the robot will remind himself when he gets home to activate the thought: “call the boss”.  The sentence has a pattern, which is:  when you recognize your home, activate this thought: “call the boss”. 

The robot can be fickle minded as well.  The robot might decide to go to Panda express for lunch, but when he gets there he decides to go to Burger King, next door instead. 

Deciding on tasks, like I said before, is based on at the moment.  If the robot is a student in a college studying computer science, and the professor gave him a software assignment, then the robot will plan when he will write the software program.  The professor might give the students 2 weeks to finish the software program.  The robot knows that he has lots of time so he decides to write the software program tomorrow.  If the robot is writing the software program and he has to sleep, he will plan the task of continuing to write the software program tomorrow in his free-time.  If the robot is writing the software program for 5 straight hours and he is tired, he will decide to take a 1-hour break, before continuing his work.  Within the 1-hour break, the robot will use logic to find something to do to relax, such as deciding on watching a movie or playing videogames. 

In other cases, the robot might be in trouble and he has until tomorrow to finish the software program.  He will decide to stay up all night to finish the software program.  This task will defeat any pre-existing task in the task container, such as sleeping at 9 pm or eating a sandwich before sleeping. 


Rules in the rules container

When the robot gets up each day, knowledge in terms of rules will pour into the rules container.  These rules dictate the boundaries of certain behavior.  In the identity container, the robot has many rules that it follows.  These rules include:  likes, dislikes, goals, limits, how to do things, what rules that must be followed while doing tasks and so forth. 

Since there are soo many rules in life, the robot has to follow rules at the moment.  That’s where the intelligent pathways are needed.  When the robot predicts the future, there are actually “event pools” stored at certain times when events happen.  In the event pool are rules ranked in order that activates in the robot’s conscious.  The computer program in the robot’s brain will manage these rules and determine what rules to activate at certain times. 

If the robot was playing baseball, and he is batting, the rules of conduct (regardless of how complex they may be) will be populating the rules container.  The computer program will manage these rules and activate the most relevant for that given situation.  Most of these rules are just sentences that have discrete math properties such as if-then statement, for-loops, while-loops, functions, encapsulated rules and so forth.  The most famous is the if-then statements.  If the robot recognizes this situation then activate these set of rules.  

Finally, the task container and the rules container go hand in hand.  The robot accomplishes tasks based on the rules.  These are the rules that you must follow in order to accomplish this task.  These set of rules are needed in order to solve a math equation.  These set of rules are needed in order to play chess.  The rules are boundaries set that the robot must follow in order to accomplish his objectives.   


Deciding to do a task

The decision maker is ultimately the robot and he controls what decisions to make.  The intelligent pathways in memory make decisions and extract relevant data from memory to make decisions.  Because of this, the robot must have information about its identity (FIG. 43).  In the identity container, there exists information that is important to the robot.  Things like dislikes and likes are located there. 

Depending on the situation, the data in the identity container varies.  If the robot is shopping for groceries at the local supermarket, then likes and dislikes of food will be stored there.  If the robot was in a clothing department, then likes and dislikes of clothes will be there.  When the robot consciously decides to buy groceries from the supermarket, information about food and possible purchases pour into the robot’s conscious like a facet valve.  The more he thinks about buying groceries the more information is extracted from memory.

FIG. 43 

The robot is alive because of all the containers in the conscious.  The robot can control what tasks and how to carry out each task.  The robot controls the rules that it must follow in order to fulfill the tasks.  The identity container contains the likes and dislikes of the robot and what its values are when making decisions.  The robot is self-aware because of the control of itself.  External objects like people and the environment influences the robot to make decisions, but it’s the robot that ultimately makes the decisions. 

The identity container also establishes relational links with other currently encountered objects.  Family members are established and friends/foes are established.  The things that the robot possesses are known in the identity container.  Things like the robot has 2 arms, 2 legs, a brain, two eyes, 1 mouth, 2 ears and so forth.  He knows that the visual images coming into the eye belongs to him and the things that he hears from the environment come from his ears.  The robot knows that he can control most of the physical aspects of its body, such as focusing the eyes on desired areas, focusing the ears to hear certain sound, thinking, moving its body and so forth.

If the robot has money in his pocket, he knows that the money belongs to him.  Possession is a learned thing by teachers.  Teachers will use sentences to associate how one object can possess another object.  If the robot pays money for a domain name (which is a non-physical object), the robot knows that he possesses the domain name.  The sentence best describe possession:  “if the robot pays for the domain name, then the domain name belongs to the robot”.  The word belong is a word that has hidden values that mean possession.

The robot is taught at a very young age common sense knowledge and what decisions benefit itself.  Simple decision making are taught to the robot at its early stages.  A lesson like selecting between two items based on price is taught.  The teacher gives the robot two identical ice cream and asks the robot to select one.  One cost 2 dollars and the other cost 8 dollars.  Using common sense, the robot will determine that both ice creams are identical, but one is more expensive than the other.  The teacher forces the robot to select the cheaper ice cream. 

The teachers are there to supervise the learning by guiding the robot with sentences.  These sentences tell the robot what is right and what is wrong.  If the robot selects the 8 dollar ice cream, the teacher will ask the robot why.  At that point, he/she will say:  “that is wrong, you have to select the cheaper ice cream”.  The teacher will also explain why.  The teacher will repeat the lesson until the robot selects the correct ice cream and makes the right decision.

Learning will be gradual, whereby the teacher will teach more complex decision making examples.  For example, the teacher might display 5 different chilli can and the prices for each can are different.  The robot has to select one of the 5 chilli cans.  The prices have to be compared, the presentation of the chilli cans have to be compared, the amount of money the robot currently has, and the brands of the chilli have to be compared. 

In the identity container contains a very important fact:  “the robot is allergic to onions”.  If 1 of the chilli cans has onions in it, the robot should rule that one out.  The robot doesn’t care if the chilli can is cheap or the name-brand is well known.  The fact that he can’t eat the chilli tells the robot that he won’t be buying that chilli can.  Thus, the teacher will teach the robot to rule out possible chilli cans.  Next, the teacher will teach the robot to compare and narrow down the choices to 2 or 3.  In order to do this, a sophisticated form of logic is required to compare prices and name-brands.  Facts about the 4 remaining chilli cans are listed below.


1.  $2.50                2.  $10.95                 3.  $3.95                4.  $1.99

well-known               best brand                well-known            unknown


The robot will know that he currently has 5 dollars in his pocket.  Chilli can 2 will be ruled out because he can’t afford it.  The next step is to compare the other 3 chilli cans in terms of price and name-brand.  The robot likes the $3.95 choice, but only slight above the other two choices.  The robot likes the $1.99 choice because its cheap, but dislikes the taste of the chilli.  After an elaborate debate, the robot decided to buy the $2.50 because it is considered not too cheap and not too expensive.  He also likes the taste of the chilli. 

Decision making is a personal thing, but the teacher is there to help the robot make the best decision and to guide him in the decision making process.  If I were to select from the 4 choices above, I would probable pick the $1.99 choice because I like to buy cheap things.  Buying cheap is one of the rules that I follow when I buy anything.

In another example, the teacher can put 10 fruits on a desk and ask the robot to select one fruit.  In the robot’s memory all 10 fruits are stored there and each fruit has powerpoints to determine its desirability.  The decision at this point is to search for each fruit in memory and compare their powerpoints.  The robot might be taught to select 3 fruit out of the 10 quickly and then compare the 3 fruit in terms of different aspects to ultimately decide on 1 fruit.  When the robot does select 3 fruits, he will have to compare aspects of the fruits such as size and taste.  Let’s say the robot selects a banana, an apple and an orange.  The banana is quick to eat and taste good.  The apple last longer, but it doesn’t taste good.  The orange last long and it taste good.  Based on these three logical thoughts, the robot will determine which of the 3 will benefit itself.  It’s obvious from the 3 logical thoughts that the orange is a good choice because it taste good and it last long. 

Thus, decision making is simply generating facts about each object and selecting among these facts.  Comparing facts can be used as well, such as:  “this chilli can is larger in size than that chilli can”, “this chilli can’s brand-name is popular and the price is cheap”, “this chilli can is expensive, but the chilli is very good”.  Based on these logical facts generated by the items for selection, the robot has to determine which item is the best.  In some cases, a decision is a debate in the robot’s brain. 

Another way that decision making is learned is by doing grade school worksheets.  Decision making worksheets are given to the robot and he has to accomplish the assignment.  The robot will be graded on how well it makes decisions.  If the robot has a good grade, that means he made the right choices.  On the other token, if the robot has a bad grade, that means he made terrible choices.  The teacher will correct the errors in front of the robot so that he can see what mistakes he made. 

Yet another way of learning decision making is by watching TV shows like the price is right.  People are put on the stand to make decisions.  Some decisions are based on logic while others are based on random selection.  If the robot watches 10 scenes and in each scene the task was to select from 3 random doors, the robot will copy the average behavior of the contestants.  For example, 7 out of 10 times the contestants selected the 2nd door, then the next time the robot has to select a random door, he will select the 2nd door. 

FIG. 44 is a diagram showing input/output again.  The robot has to find out the pattern between the input and the output.  In this case, the input is the question and the output is the selection.  When the robot watches people decide on things, he tends to mimic their behavior. 

FIG. 44

Decision making can also be corrected by pain and pleasure.  What if the robot buys a chilli can from the supermarket and found out that the chilli had tiny onions in it?  The robot will remember the brand and never buy from this brand again.  The pain of finding out the chilli he bought had onions in it caused the robot to put negative points on this type of chilli. 


Tasks to do and rules to follow

When the robot makes a decision, that task will be inserted into the task container (FIG. 43).  If the robot decides to eat lunch at Subway, it will insert instructions, via sentences, into the task container.  The computer program inside the conscious keeps track of the tasks and remind the robot when to do these tasks.  The computer program generates an operating system to manage tasks.  Things like listing linear tasks, solving interruption of tasks, rearranging tasks, executing multiple simultaneous tasks, reminding when to do tasks, erasing tasks, modifying tasks, managing hierarchical tasks are all done by the computer program inside the robot’s conscious.  

Distant tasks can also be inserted into the task container.  If the robot decides one day that he will take a vacation next week Monday, then he will store that task in the task container.  Tasks in the task container are not fixed and can be changeable.  For example, if Monday comes and the robot is too busy, he can procrastinate and say that he will take the vacation next month. 

Tasks can come from both external commands or internal commands.  A teacher telling the robot to do a task is considered external command and logical thoughts to do tasks is considered internal commands.


Playing chess example

Traditional chess AI programs rely on generating next step moves and ultimately selecting one move to play the game.  A human being doesn’t play chess like an AI chess program.  In fact, the reason the best chess player in the world lost to a computer is because the AI chess program was cheating.  The human being can only see 2-3 steps into the future.  The AI chess program can see 300 steps into the future for all pieces. 

I will explain how the robot plays chess using human level artificial intelligence.  The reader can compare my methods with the traditional AI chess programs and note the dissimilarities. 


FIG. 45 is a diagram depicting the intelligent pathway that the robot will follow to play the game of chess.  The first step is to consciously decide on playing chess.  The robot determines that the opponent is an expert.  This prompts the robot to play the game with great attention.  Not only will the task of playing chess be in the task container, but the other task is to play the game with great attention. 

The next thing that happens is that knowledge will pour in from memory into the conscious and populate the containers.  The rules of the game are poured in, the objectives of the game are poured in and the strategies to use are poured in.  The fact that a data is a task and another data is a task groups them together and stored in the task container.  The fact that a data is a rule and another data is a rule groups them together and stored in the rules container.  Thus, data coming into the conscious is organized based on association.  The data should be structured in a hierarchical manner so that only important data enters the conscious. 

As the robot play chess during runtime, the intelligent pathways selected will define the computer program in the conscious to play the game.  Every action is based on at the moment.  The only exceptions are the rules of chess.  The rules container contains rules that might or might not happen.  These rules are extracted because the robot’s brain thinks that the data might be needed in a given situation.  One of the rules to follow is that the player (the robot) and the opponent have to take turns when playing the game.  Each can only make one move per turn.  This rule is followed by the robot throughout the game. 

The computer program will do things for each given moment in time.  When the opponent moves, the robot will plan strategies and guess the opponents next move.  When it’s the player’s turn, he has to plan several future steps and select only one to act.  The player knows that one of the rules of playing chess is to make your move quickly.  Thus, the robot has a time limit to make his move. 

The intelligent pathways will generate an optimal computer program to manage tasks for the game of chess.  The game will continue and the player and opponent will keep taking turns until one wins. 

One of the strategies used to play chess is to predict the future by imagining a fabricated movie of 3 future chess moves (one future move per chess piece).  The robot has to fabricate 3 future chess piece and decide which of the 3 moves to select.  The method to do this is to imagine the pieces moving in this space or that space.  While the imagining takes place the robot is also imagining possible opponent moves.  The brain manages these fabricated future movies and selection of each future move is done by comparing with other fabricated future moves.

My guess is that the brain fabricates one future move and it fabricates another future move, compare the two fabricated moves to determine which is better, then selecting one of the two future moves.  Next, the brain will fabricate another future move compare it with the previous selected future move and determine which is better.  Previous unselected future moves can be reconsidered too.  The robot’s brain will do this until he has to act. 

Each human player uses their own personal methods, but most chess players use the method described above.  Some expert players can actually fabricate 3 step moves for each chess pieces and determine which is better. 

While the robot’s brain is managing the task of fabricating future moves, it will also follow the rules of chess.  If the robot is trying to fabricate a future move, he has to first identify the chess piece and determine how it can and can’t move.  The horse piece can only move in an L movement.  This rule will activate at the moment the robot wants to fabricate a future move using the horse piece.  If the chess piece is the pawn, he can only move one step forward.  While making decisions on which fabricated future move is better, the robot has to consider the worth of the chess piece.  The king is a very important piece because it can do all the moves that any of the other pieces can do.  The pawn is the weakest because it can only move forward.  During the decision making process, the robot has to weigh all the fabricated future moves and only decide on one action.  The computer program inside the conscious will also remind the robot he has a limited time period to decide.   

Playing chess is a lot like shopping for groceries.  There is an internal debate that determines which logical thought to choose.  Lessons from teachers guide you to decide on things.  

In terms of learning how to play chess, a teacher or a friend will teach you how to play the game.  He/she might tell you this:  “imagine several future moves and select only one”.  However, the learning of chess comes from a universal pathway that can play any game.  The teachers have taught you to teach yourself how to play an unknown game.  When you don’t know how to play a game you have to first learn the rules and objectives.  This can be done by reading instruction manuals or having a teacher teach you. 

Learning the objectives and rules of a game is one thing, but going through trial and error to determine what strategies are good and bad is another.  Trial and error makes the robot more skilled at the game.  Trial and error will keep the good strategies and delete the bad strategies.  The robot can also be taught to generate strategies and try them to see if they are good or bad strategies. 

Also, teachers can teach the robot to identify bad things or good things in the game.  Having lots of chess pieces on the board for the player is a good thing.  Having less chess pieces on the board is a bad thing.  Having strong chess pieces on the board is a good thing and having weak chess pieces on the board is a bad thing.  By understanding the bads and goods of the game, the robot can learn to be better in the future. 

Playing chess isn’t a very good example to use in terms of identifying bad and good things in a game.  If you play videogames, it is easy to identify the bads and the goods.  When a player dies, that is considered bad.  That bad move will give the player pain, which is necessary to improve in the future.  Bad things in a videogame will include:  dying, losing life energy, missing a powerup, getting hit by an enemy, jumping into a lake, losing the game, not having high scores and so forth.  The good things in a videogame will include:  having high energy level, gaining powerups, avoiding enemies, defeating enemies, having high scores, winning a level, beating the boss, hurting a boss and so forth. 

If you play videogames, the identifying of good and bad things is a vital part of the game.  If the player does good things, it will steer the game in the right direction.  If you do bad things the game will steer the game into the wrong direction.  The robot should also have the knowledge to identify the objectives of the game.  In some unknown games, I have to play many times before I logically identify the objectives.  For example, in donkey kong, I didn’t know what the objective of the game was at first.  I tried something and died, then I tried something else and I died again.  However, by trying something and identifying what is good and bad in the game, I was able to use logic to identify the objectives.      

The point I’m trying to make is that when the robot has to play an unknown videogame, the identifying of good and bad things in the game will help it generate better strategies to play the game.  If the robot has good strategies to play the game, then he will eventually succeed in passing all the levels (refer to my previous books and patent applications regarding how the robot plays videogames such as contra and the legend of zelda).


<< content               next chapter >>  



Home | HLAI | UAI | Books | Patents | Notes | Donation

Copyright 2006 (All rights reserved)