{"id":11880,"date":"2023-04-08T19:09:44","date_gmt":"2023-04-08T19:09:44","guid":{"rendered":"https:\/\/aurelis.org\/blog\/?p=11880"},"modified":"2023-04-09T08:19:11","modified_gmt":"2023-04-09T08:19:11","slug":"sequential-problem-solving-with-partial-observability","status":"publish","type":"post","link":"https:\/\/aurelis.org\/blog\/artifical-intelligence\/sequential-problem-solving-with-partial-observability","title":{"rendered":"Sequential Problem Solving with Partial Observability"},"content":{"rendered":"\n<h3>My goodness! Please hang on. Against all odds, this may get interesting. Besides, it\u2019s about what you do every day, all day long.<\/h3>\n\n\n\n<p><strong>This is also what many would like A.I. to do for our sake.<\/strong><\/p>\n\n\n\n<p>Even more, it is what artificial <em>intelligence<\/em> is about. Contrary to this, what is called A.I. these days is different in three respects:<\/p>\n\n\n\n<ul><li>Present-day A.I. is good at one-shot performances, not sequential processing following a moving (sub)target with underlying consistency.<\/li><li>It is an automation of decision-making, not problem-solving.<\/li><li>The \u2018partial\u2019 of the A.I. of today is quite limited.<\/li><\/ul>\n\n\n\n<p>You may notice that all three respects are relative. There are no clear-cut borders, and present-day A.I. is slowly creeping up on the three. Nevertheless, together they form a workable distinction between \u2013 if you like \u2013 the presence or absence of \u2018intelligence\u2019 as a concept. This is interesting to a pragmatical mind.<\/p>\n\n\n\n<p>It\u2019s not the end of the story, but it may be the end of the beginning. Therefore, let\u2019s delve a bit into each element, then come back to the whole and wrap up, including an admonition.<\/p>\n\n\n\n<p>[For insiders: What I find missing most in this \u2013 to avoid making things more challenging \u2013 is potent complex function approximation \u2015 in one word: \u2018depth,\u2019 as in that specific contribution of neuronal networks.]<\/p>\n\n\n\n<p>[For non-insiders: Interesting, isn\u2019t it, how the same description can tightly apply to the human and artificial case?]<\/p>\n\n\n\n<p><strong>Sequential<\/strong><\/p>\n\n\n\n<p>This may be most advanced in present-day robotics, a field, however, with relatively little progress. In research, at least, the field of reinforcement learning \u2013 taking on the sequential challenge \u2013 is progressing rapidly, even booming. It\u2019s still a minor subfield in the whole of the A.I. domain, but this may change dramatically in the near future with a host of practical applications.<\/p>\n\n\n\n<p>We see then the emergence of systems (<a href=\"https:\/\/aurelis.org\/blog?p=11625\">agents<\/a>) that can independently form a strategy (or \u2018policy\u2019) on the basis of experiences, going from state to state in the state space that forms the environment. A policy is a mapping from observations to actions, balancing immediate and long-term goals. For instance, human coaching happens (or should happen) on the basis of keen strategies following specific requirements as well as possible, avoiding biases of many sorts. You may recognize in this the endeavor of <a href=\"https:\/\/aurelis.org\/blog\/category\/lisa\">Lisa<\/a>.<\/p>\n\n\n\n<p><strong>Problem-solving<\/strong><\/p>\n\n\n\n<p>The main difference with decision-making lies in flexibility \u2015 or, from a different take, complexity.<\/p>\n\n\n\n<p>To make a simple decision, the elements are already present.<\/p>\n\n\n\n<p>For problem-solving, the elements may need to be sought, balancing their gathering and utilization. While doing so, even the problem domain may change. It\u2019s a sign of intelligence if a person can change the problem domain when applicable instead of trying too hard to solve the problem within what has been given. We call that \u2018insight,\u2019 and it\u2019s a sign of intelligent flexibility in thinking. A genius may even develop a broad original insight that immediately makes problem-solving much more feasible for others. This is a genuine act of discovery.<\/p>\n\n\n\n<p><strong>Partial observability<\/strong><\/p>\n\n\n\n<p>For example, an artificial image recognizer may do the job under conditions of partial observability quite well \u2015 in many cases, already to a supra-human level. Many medical opportunities are around the corner or already achieved in research. What we see in practice nowadays is only a small part of this.<\/p>\n\n\n\n<p>The A.I. may get better at reasoning with this input, such as by understanding <em>why<\/em> an observation (part of a state of reality) is only partial and what this means to how the system can process the observation. Also, it may improve in managing partialities that it has not been specifically trained for. This includes evaluative (subjective) and sampled (to a higher or lower degree) experiences (multifactorial observations including states, actions, and feedback). The observability that the system should learn to deal with can be in reality or in tractability (according to the agent\u2019s power).<\/p>\n\n\n\n<p><strong>Wrapping up<\/strong><\/p>\n\n\n\n<p>The combination of the three makes the artificial challenge bigger. At the same time, it provides additional opportunities to be more performant than the simple sum of the parts. You can see the combination in action in anything you do as a human person. If you look closer (at this domain or yourself), you can see how the elements continually boost each other.<\/p>\n\n\n\n<p>This is the case even more than you are consciously aware of. For instance, your partial \u2018visual observability\u2019 (seeing only part of the environment sharply at each moment) doesn\u2019t strike you much because you are continually and willfully moving around. Especially your eyes are continually busy scanning, not at random but in meaningful ways also without your knowing. Your brain continually solves many problems before what you see gets interpreted by, well, you, of course.<\/p>\n\n\n\n<p>Likewise, the threesome forms a good jumping board for complex, intelligent processing in artificial intelligence. I am confident that delving into this combination (much further, of course) will lead us there.<\/p>\n\n\n\n<p>So I promised you an admonition.<\/p>\n\n\n\n<p><strong>Self-enhancing property<\/strong><\/p>\n\n\n\n<p>An artificial system that is good in the title of this text can increasingly get better in heightening its performance as to the same. Thus, it becomes self-enhancing, especially if there is also continually much relevant input from humans. This is even more true if the system can seek out input for itself and its learning needs.<\/p>\n\n\n\n<p>Challenging? Dangerous would be to fall into an <a href=\"https:\/\/aurelis.org\/blog?p=11775\">A.I.-phobia<\/a>. However, this deserves a solid admonition to follow the <a href=\"https:\/\/aurelis.org\/blog?p=2819\">Journey Towards Compassionate A.I.<\/a><\/p>\n\n\n\n<p>No time to waste.<\/p>\n<div data-object_id=\"11880\" class=\"cbxwpbkmarkwrap cbxwpbkmarkwrap_no_cat cbxwpbkmarkwrap-post \"><a  data-redirect-url=\"https:\/\/aurelis.org\/blog\/wp-json\/wp\/v2\/posts\/11880\"  data-display-label=\"0\" data-show-count=\"0\" data-bookmark-label=\" \"  data-bookmarked-label=\" \"  data-loggedin=\"0\" data-type=\"post\" data-object_id=\"11880\" class=\"cbxwpbkmarktrig  cbxwpbkmarktrig-button-addto\" title=\"Bookmark This\" href=\"#\"><span class=\"cbxwpbkmarktrig-label\"  style=\"display:none;\" > <\/span><\/a> <div  data-type=\"post\" data-object_id=\"11880\" class=\"cbxwpbkmarkguestwrap\" id=\"cbxwpbkmarkguestwrap-11880\"><div class=\"cbxwpbkmarkguest-message\"><a href=\"#\" class=\"cbxwpbkmarkguesttrig_close\"><\/a><h3 class=\"cbxwpbookmark-title cbxwpbookmark-title-login\">Please login to bookmark<\/h3>\n\t\t<form name=\"loginform\" id=\"loginform\" action=\"https:\/\/aurelis.org\/blog\/wp-login.php\" method=\"post\">\n\t\t\t\n\t\t\t<p class=\"login-username\">\n\t\t\t\t<label for=\"user_login\">Username or Email Address<\/label>\n\t\t\t\t<input type=\"text\" name=\"log\" id=\"user_login\" class=\"input\" value=\"\" size=\"20\" \/>\n\t\t\t<\/p>\n\t\t\t<p class=\"login-password\">\n\t\t\t\t<label for=\"user_pass\">Password<\/label>\n\t\t\t\t<input type=\"password\" name=\"pwd\" id=\"user_pass\" class=\"input\" value=\"\" size=\"20\" \/>\n\t\t\t<\/p>\n\t\t\t\n\t\t\t<p class=\"login-remember\"><label><input name=\"rememberme\" type=\"checkbox\" id=\"rememberme\" value=\"forever\" \/> Remember Me<\/label><\/p>\n\t\t\t<p class=\"login-submit\">\n\t\t\t\t<input type=\"submit\" name=\"wp-submit\" id=\"wp-submit\" class=\"button button-primary\" value=\"Log In\" \/>\n\t\t\t\t<input type=\"hidden\" name=\"redirect_to\" value=\"https:\/\/aurelis.org\/blog\/wp-json\/wp\/v2\/posts\/11880\" \/>\n\t\t\t<\/p>\n\t\t\t\n\t\t<\/form><\/div><\/div><\/div>","protected":false},"excerpt":{"rendered":"<p>My goodness! Please hang on. Against all odds, this may get interesting. Besides, it\u2019s about what you do every day, all day long. This is also what many would like A.I. to do for our sake. Even more, it is what artificial intelligence is about. Contrary to this, what is called A.I. these days is <a class=\"moretag\" href=\"https:\/\/aurelis.org\/blog\/artifical-intelligence\/sequential-problem-solving-with-partial-observability\">Read the full article&#8230;<\/a><\/p>\n<div data-object_id=\"11880\" class=\"cbxwpbkmarkwrap cbxwpbkmarkwrap_no_cat cbxwpbkmarkwrap-post \"><a  data-redirect-url=\"https:\/\/aurelis.org\/blog\/wp-json\/wp\/v2\/posts\/11880\"  data-display-label=\"0\" data-show-count=\"0\" data-bookmark-label=\" \"  data-bookmarked-label=\" \"  data-loggedin=\"0\" data-type=\"post\" data-object_id=\"11880\" class=\"cbxwpbkmarktrig  cbxwpbkmarktrig-button-addto\" title=\"Bookmark This\" href=\"#\"><span class=\"cbxwpbkmarktrig-label\"  style=\"display:none;\" > <\/span><\/a> <div  data-type=\"post\" data-object_id=\"11880\" class=\"cbxwpbkmarkguestwrap\" id=\"cbxwpbkmarkguestwrap-11880\"><div class=\"cbxwpbkmarkguest-message\"><a href=\"#\" class=\"cbxwpbkmarkguesttrig_close\"><\/a><h3 class=\"cbxwpbookmark-title cbxwpbookmark-title-login\">Please login to bookmark<\/h3>\n\t\t<form name=\"loginform\" id=\"loginform\" action=\"https:\/\/aurelis.org\/blog\/wp-login.php\" method=\"post\">\n\t\t\t\n\t\t\t<p class=\"login-username\">\n\t\t\t\t<label for=\"user_login\">Username or Email Address<\/label>\n\t\t\t\t<input type=\"text\" name=\"log\" id=\"user_login\" class=\"input\" value=\"\" size=\"20\" \/>\n\t\t\t<\/p>\n\t\t\t<p class=\"login-password\">\n\t\t\t\t<label for=\"user_pass\">Password<\/label>\n\t\t\t\t<input type=\"password\" name=\"pwd\" id=\"user_pass\" class=\"input\" value=\"\" size=\"20\" \/>\n\t\t\t<\/p>\n\t\t\t\n\t\t\t<p class=\"login-remember\"><label><input name=\"rememberme\" type=\"checkbox\" id=\"rememberme\" value=\"forever\" \/> Remember Me<\/label><\/p>\n\t\t\t<p class=\"login-submit\">\n\t\t\t\t<input type=\"submit\" name=\"wp-submit\" id=\"wp-submit\" class=\"button button-primary\" value=\"Log In\" \/>\n\t\t\t\t<input type=\"hidden\" name=\"redirect_to\" value=\"https:\/\/aurelis.org\/blog\/wp-json\/wp\/v2\/posts\/11880\" \/>\n\t\t\t<\/p>\n\t\t\t\n\t\t<\/form><\/div><\/div><\/div>","protected":false},"author":2,"featured_media":11881,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","jetpack_publicize_message":""},"categories":[28,30],"tags":[],"jetpack_featured_media_url":"https:\/\/i1.wp.com\/aurelis.org\/blog\/wp-content\/uploads\/2023\/04\/2078.jpg?fit=961%2C560&ssl=1","jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p9Fdiq-35C","jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/aurelis.org\/blog\/wp-json\/wp\/v2\/posts\/11880"}],"collection":[{"href":"https:\/\/aurelis.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/aurelis.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/aurelis.org\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/aurelis.org\/blog\/wp-json\/wp\/v2\/comments?post=11880"}],"version-history":[{"count":7,"href":"https:\/\/aurelis.org\/blog\/wp-json\/wp\/v2\/posts\/11880\/revisions"}],"predecessor-version":[{"id":11889,"href":"https:\/\/aurelis.org\/blog\/wp-json\/wp\/v2\/posts\/11880\/revisions\/11889"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/aurelis.org\/blog\/wp-json\/wp\/v2\/media\/11881"}],"wp:attachment":[{"href":"https:\/\/aurelis.org\/blog\/wp-json\/wp\/v2\/media?parent=11880"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/aurelis.org\/blog\/wp-json\/wp\/v2\/categories?post=11880"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/aurelis.org\/blog\/wp-json\/wp\/v2\/tags?post=11880"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}