Information

Overview

The coursework for this module is based around (1) designing intelligent autonomous agents and an environment with which they interact, (2) setting those agents a task, (3) asking one or more questions about that task, and (4) evaluating it using experimental methods. You will then present the results from this in a report, video or podcast, which will also explain the context for the work; and, submit the code and other supplementary materials. Students doing COMP4105 will in addition do a short presentation explaining how their coursework relates to contemporary research and technology.

Requirements & Specifications

An autonomous intelligent agent is a program that operates in a particular environment, perceives aspects of that environment, and then uses its intelligence to choose actions that change that environment, to carry out some task. Typically, these actions are a mixture of  responses (either immediate or deliberated) to its perception and memory, and proactive actions such as exploration.

Your task for this coursework is to design an agent-based system containing the following four aspects:

An Environment. This is the (virtual) place where the agents will operate. It could be one of:

•    A simulation of a physical environment in which mobile robotic agents move. This could be the simulation used in the classes earlier in the semester (perhaps extended), a robot environment such as The Player Project

(http://playerstage.sourceforge.net), or a project in Unity or a similar game environment if you are familiar with one from elsewhere.

•    A language environment, for example where humans are interacting using written or spoken language with one or more intelligent agents,

•    The Bristol Stock Exchange system

(https://github.com/davecliff/BristolStockExchange) or a similar simulation of some aspect of the economy or society

•    A game environment such as Ms. PacMan (https://gym.openai.com/envs/MsPacman- v0/), the Open Racing Car Simulator (http://torcs.sourceforge.net), RoboCup

(https://www.robocup.org/leagues/23) or similar (see e.g. http://www.gvgai.net)

•    One of the more complex task environments from the OpenAI Gym (https://gym.openai.com)

There is no need to develop the environment yourself—the focus of the project will be on the agents in the environment (robots, trading agents, game-playing agents, autonomous drivers, language agents, etc.) – but it is likely that you will set up the details of the environment to address your specific question. You are allowed to use the code from the classes, but please try to make it clear which parts of the code are taken from the class examples, and which is your own work (we appreciate that this is sometimes complicated to do at a line-by-line level, but you should indicate this in terms of which functions/classes etc are taken without modification and which are modified and to what extent).

Autonomous Agents. You should introduce one or more autonomous intelligent agents into the environment, which use some kind of AI to solve a task.

•    Examples of AI could be an AI planning system such as Goal Oriented Action

Planning (http://alumni.media.mit.edu/~jorkin/goap.html), a search algorithm such as A* search, a genetic or swarm search, a reinforcement learning algorithm, fuzzy logic, or a hard-coded reactive or state-machine AI.

•    The task will be one relevant to the environment: e.g. a robot vacuum cleaner

clearing up dirt, a poet trying to write verse about the day’s news, a trader trying to optimise its returns, a game player trying to get a high score in a game, etc.

Within reason, you can use any language to do this. If you are planning to use anything other than Python, Java, C/C++, MATLAB/Octave, R, JavaScript, and mainstream web technologies such as HTML/CSS/JS, then please mention this in your topic approval.

A Question. You should ask a specific question (or a set of related questions) about your system. For example:

•    How do different approaches (e.g. a genetic algorithm, an A* search algorithm, a

hard-coded heuristic, different kinds of reinforcement learning algorithms) compare in terms of task performance?

•    How does the performance of the system change as we vary the number of agents in it?

•    If the system is trained on one version of the environment, does that learning transfer over to a new version of the environment?

•    How do different kinds of communication/coordination between agents affect the efficiency of those agents on the task?

•    How much improvement does storing some information (e.g. a map of the

environment) make compared to carrying out the task in a purely reactive way?

•    How do different kinds of sensing/perception systems affect the capacity of the agent to carry out its task?

•    How sensitive is the agent to error/noise?

A Set of Experiments. You should answer your question by carrying out a set of

experiments. Remember the structure that we talked about in the lecture on 17th  February:

1.   Implement code that carries out a run of the agent’s behaviour and measures performance

2.  Then, run that code multiple times to get a measure of average performance

3.  Then, repeat that process for the different conditions in your question, and use descriptive statistics, charts/visualisation, and/or inferential statistics (e.g.

significance tests) to test your question

Then, you should discuss the question using these experimental results as your evidence.

If this evaluation involves asking people to interact with your code, then please read the notes on research ethics that are in the Coursework section of the module Moodle page.

Constraints. Your project is expected to demonstrate 90-100 hours of effort. So, you should not just run a basic reinforcement learning or convolutional neural networks library/tutorial on a simple environment from AI Gym or similar, though you can use these as part of your work.

You are not allowed to develop a basic chatbot i.e. a single system that interacts with a human using intent matching, identity management, transaction processing, question answering etc. If you want to do something on language agents, you should do something  that goes beyond this (e.g. a creative language system, a system that involves interactions between multiple language agents, or a system that involves a language-based interface to another system). If you have doubts about this, please mention it in the topic approval.

Getting Started

You are at liberty to use any readily available libraries, frameworks, APIs, services, datasets etc. as you want—but your work must demonstrate substantial work in using, combining and building on what you use. You are not permitted to simply re-submit coursework from another module, but you could potentially build on code that you have written previously and build on it in a new way. If your work extends or re-uses coursework from other modules, it must make a substantial and distinctive contribution beyond what was in those courseworks, and you must clearly identify what the new contribution is.

The key to getting a good grade on this module is to do a piece of work that is balanced between all parts of the coursework—you should justify why the topic is important and perhaps explore why it is a good model problem for AI/autonomy, provide clear context in research in the area, explain and justify the design of the agents and environment, have clear questions, make sure that the experiments that you do are focused on the questions, present the results clearly using statistical and/or visual methods, and make sure that you reflect on the questions using the evidence from the experiments, and finally put the work  into the wider context of AI, agents and autonomous systems.

Recommendations

Here are a few examples of things that you could do. You don’t have to do one of these— indeed, we would prefer you to come up with your own idea—but, these would all be acceptable project ideas if you want to do them:

•    To take the “robot vacuum cleaner” from the early classes, and experiment with

different numbers of robots, and different coordination strategies (e.g. robots try to stay a fixed distance from each other, compared to sharing a map that they build up)

•    Explore how multiple language agents, each of which has a particular set of

knowledge or perspective can, discuss/argue with each other to explore a topic.

•    Take a number of different trading strategies and run them in the Bristol Stock

Exchange system with varying amounts of noise/uncertainty, to see how robust each strategy is.

•    Develop a game-playing agent or a control agent for an NPC or opponent in a game, and explore different strategies for playing (or learning to play) a game.

•    Take the “avoid the cats” problem from the class, and compare a number of

strategies for the problem: warning the cats vs. moving out of the way, and learning when to act based on a simple statistical approach vs. a decision-tree approach.

For students on COMP4105, the portfolio of work (including the report/video/podcast) will count for 90% of the marks on the module and the presentation for 10%.

Assessment Criteria

Learning Outcomes

This assessment tests the following learning outcomes:

•    To describe the problems and techniques in the design of intelligent agents, explain common agent architectures.

•    To evaluate agent requirements and specifications. To analyse agent behaviour in a variety of environments.

•    To demonstrate advanced Al programming skills.

•    To analyse and design complex multi-component systems.

•    To present work in a professional manner using appropriate technical concepts and vocabulary (including both written and, for COMP4105 only, spoken presentation).

Marking Guide

Marking will take into account:

•    The intrinsic complexity of the overall project

•    Background research and how you have used it to contextualise your work

•    The choice of task environment and how you have used it/adapted it for your specific project

•    The effective use of artificial intelligence and agent-based systems ideas from the course and your wider studies in designing your autonomous agents

•    How clear your question(s) are, how well the experiments have been designed to answer them, and your level of rigour in planning and analysing the experiments

•    How well the report answers the question by using the evidence from the experiments

•    The overall clarity and structure of the report, appropriate use of scientific and

technical English, and the quality of charts, diagrams, pseudocode where relevant

•    The quality of reflection on the successes and limitations of the work

•    (For students doing a presentation) the structure of the presentation, the clarity of explanations, and good use of slides or other visual aids

There is a detailed marking scheme at the end of the document.