Help-Desk Systems Stephen Lee-Urban November 17, 2006

Help-Desk Systems Stephen Lee-Urban November 17, 2006

References Experience Management: Foundations, Development Methodology, and InternetBased Applications, Ralph Bergmann. – Chapter 9: Developing and Maintaining Experience Management Applications – Chapter 11: Experience Management for SelfService and Help-Desk Support 2

Outline Motivation / Goals / Background The HOMER architecture Developing & Managing EM* Applications Evaluation of HOMER *EM Experience Management 3

Motivation Complexity of technology increasing – Operation, Maintenance, Repair – Probability of failure grows exponentially with complexity – Expertise in all features unreasonable Problem solving experience distributed – Experience overlap is variable Experience Management System (EMS) 4

Goals of EMS Create knowledge repository – Store problem solving experience – Simplify trouble-shooting complex technical domain – Domain is likely to change over time Use knowledge repository – By varying levels of expertise – In time-critical situations 5

Experience Management vs CBR (Organization) Problem acquisition Experience base Reuserelated knowledge Experience presentation Experience adaptation BOOK CBR Experience evaluation and retrieval Development and Management Methodologies Experience Management Complex problem solving Case Library 1. Retrieve 4. Retain Background Knowledge (IDSS) Slide: Dr. Munoz-Avila 3. Revise 2. Reuse 6

Help Desk Support Levels Key-user CBR Help-Desk sys. Specific technician Vendor Bottlenecks at each level. – Problems recur. – Frustration for all parties, and wasteful of resources! “Managerial” Aids to Help-Desk: – Inventory systems, trouble-ticket tools, call-tracking software – Don’t support diagnosis nor serve as knowledge repository http://www.simpsonstrivia.com.ar/simpsons-photos/wallpapers/homer-simpson-wallpaper-brain-1024.jpg 7

Outline Motivation / Goals / Background The HOMER architecture Developing & Managing EM* Applications Evaluation of HOMER 8

The HOMER System German: “HOtline Mit ERfahrung” Developed as part of INRECA-II project For CAD/CAM help-desk at DaimlerChrysler in Sindelfingen Generic experience management architecture & set of related tools – Stores experience in CB for access, reuse and extension – Drastically decreases problem solving time http://www.aeria.phil.uni-erlangen.de/photo html/portraet/griechisch/dichter/homer/homer3.JPG http://gattaca.com.ar/weblog/wp-content/homer.gif 9

Structure and Representation Determines experience management approach HOMER uses an object oriented approach to model experience. Advantages of OO approach: – Structure of system to diagnose is representable in detail – Symptoms (attr.) easily related to owning object – Semantics of prob. description can be captured and used for selecting appropriate prior exp. – High retrieval accuracy can be achieved – Particularly suited for diagnosis help-desks w/ complex equipment Can show many hard to diagnose faults – Can be used to help guide help-desk operator while describing and entering cases – Better domain model easier maintenance and use Disadvantages of OO approach: – Harder to create model – Additional effort in beginning of knowledge acquisition Representation should be discussed with system admin. Shallow? 10

Case Structure Modeled according to approach used by help-desk operators to solve problems Case: Complete path Feasible for most complex help-desks from prob. to solution Topic: problem area (e.g. hardware) Subject: failing object (e.g. printer) Behavior: way (miss-) behaves (e.g crashes) Minimum set of attributevalue pairs describing symptoms necessary to fault diagnosis. In OO, all attributes are related to certain object. Fault: problem cause Remedy: problem fix Represented as (hyper-) text 11

Experience Base Partitioning Domain size & complexity demanded accuracy & consistency partitioning Two kinds of cases: – Approved: experience reviewed by top-level experts. “Best Practice” for problem solving – Open: recently captured, pending validation (correctness, completeness, & utility) Case base separated into: – Case Buffer – all open cases – Main Case Base – all approved cases All available to helpdesk operators 12

HOMER Architecture Client-Server – Shared domain model (DM) case base (CB) – Eases maintenance of DM CB – Server accessible via intranet / internet – “Fat” client reduces network traffic Object-Oriented experience model – Not “shallow” users are experienced – For non-trivial problems 13

Lowest access rights Daily user Retrieval & acquisition Middle rights Case maint. & case approval Redundancy & consistency checks Buffer to base May modify value ranges of attributes Highest rights Creates & maint. domain and case model /- attr & concepts Admin users and rights 14

The Server CBR-Works server stores model & CB, CBR-Works modeling tools provided by server for domain modeling, case & model maintenance, & initial case acquisition Domain model snapshot 15

Attributes Help Desk Case Hierarchical domain concepts Problem, holds failure Situation, holds symptoms - Hierarchical, sub-concepts - Structure helps maint. & retrieval Loesung, holds solution Administrativa, holds organizational & statistical information 16

The Client Interface to server for case retrieval, case acquisition, & case browsing Written in Java “Hotline” component for help-desk operator – Only shows relevant information – Assists via two modes: user/system driven “Case Browser” component for experience author – For access to CB and case buffer – Revision, extension, approval of “open” cases – Removal of outdated cases 17

Hotline Component (HC) questions Used mainly during hotline operations Supports help-desk operator during problem-solving processanswers to questions Four main modes of execution problems – Problem description (initialization/refinement) solutions (cases) – Situation description (user/system driven) – Solution retrieval (manual/automatic) – Retain w/ feedback 18

HC: Problem Description question (free text) Gives initial info. on user’s problem Answer following questions values consecutively: problems – Problem’s topic? (e.g. output, software, ) – Topic’s subject? (e.g. output: plotter or printer) – Behavior? (e.g. “no output at all”) value range units 19

HC: Situation Description Problem determines relevant case structure (i.e. situation template) – Contains possible symptoms for prob. – Each symptom associated w/ question Symptom attribute values complex/simple Two modes of operation – User-driven: (no guidance, direct entry) – System-driven: Shows sorted view of most relevant questions Based on attributes with highest info gain 20

HC: Solution Retrieval Experience retrieved based on problem and situation descriptions Manual (batch) / automatic (per question) Each row a solution, sorted by relevance (similarity) Solutions viewed (read-only) 21

HC: Retain/Feedback Retain case when it – Has new experience – Requires case model extension (e.g. new value to type) Case entry interface allows – Final modifications (e.g. finish case descr.) – Annotating why op. thinks case should be retained 22

Case Browser Component question (freeCB text) Used by experience author to manage Works on both approved and open cases values Problem, situation, Can perform following operations: solution, administrativa – Case creation – Case copy – Case deletion – Case approval value range units 23

Outline Motivation / Goals / Background The HOMER architecture Developing & Managing EM* Applications Evaluation of HOMER 24

Developing & Maintaining EM Applications IT companies require efficient, effective application development – Guidelines/Methods of implementing apps – Also seek to preserve past project experience INRECA methodology – Targeted at industrial EM applications – Developed in the INCRECA-II European ESPRIT project (1996-9) 25

EM Model Problem Solving Cycle Knowledge Kernel Knowledge Acquisition & Knowledge Maintenance Technical, organizational & managerial aspects 26

Three Process Types Technical – Describe development of system & required documentation – E.g. requirements analysis, system design, implementation and testing Organizational – Address parts of business process in which software will be embedded – E.g. training end users, archiving request records, updating & maintaining the help-desk system Managerial – Provides environment and services for developing software that meets product requirements and project goals – E.g. project planning, monitoring, quality assurance 27

Methodologies Makes development an engineering activity, rather than an art Use of methodology provides benefits: – Productivity, Quality, Communication, Management decision making INRECA methodology based on software engineering principles, covering: – Project management (cost assessment, schedules, project plans, etc.) – Product/Deliverable specification – Product development and maintenance – Analysis/Organization of target env. for CBR system 28

INRECA Basic philosophy: experience based construction of experience management applications Self-application of principles of experience reuse Experience modeled as structured text documents, hyper-text linked. – Origin: Software process modeling 29

Software Process Modeling Well defined terminology to describe the engineering of a product – Process, “what”: basic step to carry out, transforming input into output. Defined by a goal, a set of alternative methods, input/output/modified products, & required resources (agent/tool) – Method (simple/complex) “how”: Detailed specification of a way to achieve a process’ goal – Product: goal of the process INRECA notation 30

INRECA Process Models Make explicit all processes, products, methods, resources and interactions Diagrams insufficient Each element must be described in detail – Called a process model – Solid basis for project planning (e.g. effort calculable based on processes involved) General vs. Specific descriptions – Common Generic/Cookbook vs. Project level 31

The INRECA Experience Base Collection of processes, products, & methods common across EM apps. Processes, products, & methods tailored to class of applications (e.g. help-desk). Each class has recipe Recipe: has process models describing how app. of that class be developed & maintained Describes experience of an instance of a completed project Project-specific info. (e.g. processes carried out, effort of processes, etc.) 32

The Common Generic Level (CGL) Overview of top-level processes and products in Bergmann, chapter 9 – General enough to occur in many projects – Problem statement “vision document” goal checklist feasibility study detailed analysis of organization project plan software development evaluation – Meanwhile, experience base can be consulted for all of the above processes Consult book for thorough exposition 33

Go / No-go (analysis & elicitation) Organizational (planning) Implementation Maintenance 34

CGL: The Three Processes Managing an AI / EM project differs from other IT projects – Concepts & technologies altogether new – Emphasize early awareness training – Ensure user-participation in iterative prototyping process Technical processes very typical of software development projects Organizational includes identifying perceived problems and opportunities at the human & organizational levels 35

Documenting in INRECA Processes, products, & methods documented and stored in experience base “Sheets” structured page w/ all relevant information in a predefined format – – – – Standardize documentation Created as web pages for easy access & use CGL has more than 150 linked sheets 8 type of description sheets: [ generic specific ] [process product simple method complex method] – Contains references to respective input, output, and modified products of the process 36

Process Description Sheets Recipe/Project Name Process Name Process Goal Input/Output/Modified Products Set of Applicable Methods (names) Agents (e.g. personnel) Required Tools Administrative Information (e.g. sheet author, version, last modified) 37

Product Description Sheets Module/Project Name Product Name (e.g. “requirements doc”) Product Description Administrative Information 38

Simple Method Description Sheets Module/Project Name Method Name Method Description Administrative Information 39

Complex Method Description Sheets Module/Project Name Method Name Method Description Details – Links to a product flow description which contains the relevant sub-processes (byproducts) Administrative Information 40

The INRECA Reuse Procedure Recipes at cookbook level are most useful for building a new application Even projects not fitting a recipe can use processes described in CGL 41

Don’t Forget to Remember! After completing project, include it in the experience base continuous improvement of the EM software development process 42

Tool Support for INRECA Experience modeling methodology tool implemented in Visio – Natural choice: shapes, hierarchical, HTML, VBA, database access Knowledge modeling tools – Integrated in the CBR-Works tool (tec:inno GmbH 1999) – CBR-Works Concept & Type Hierarchy Editor for vocabulary modeling – Similarity modeling tools integrated in the Concept & Type editors Also has an editor for adaptation and completion rules 43

Outline Motivation / Goals / Background The HOMER architecture Developing & Managing EM* Applications Evaluation of HOMER 44

Evaluation of HOMER Evaluation performed by INCRECA-II project partners to identify benefits of EMS Incoming calls monitored. Help-desk operator 1st solves conventionally, then with HOMER – No solution new case created – Two month test period: 102 calls; 45 unsuitable for HOMER 45

Evaluation (II) Of the 57 (/102) suitable problems – 18 solved (i.e. 32%) Ave. resolution time w/out HOMER: 141 min. Ave. resolution time w/ HOMER: 9 min. Correct result a limited bias HOMER sys. Mode HOMER CB transferred to a new site – Operators have no experience with the process chain so the cases are of great value – Initial knowledge acquisition gave operators insight into domain. 46

Evaluation (III) Methodology recipe created and used – Impact: productivity, quality, communication, & management decision making – Customer and developer both benefited Creation of project definition from scratch – Took three months – New development team reused recipe to define three new projects, each in one week (12x speedup!) Were sure all relevant aspects taken into account Basic recipe available developers focus on domain peculiarities quality/detail of descriptions greatly enhanced 47

Evaluation (IV) Development & testing of 1st prototype – About 6 months – Subsequent efforts: 2 weeks (13x speedup) – Less qualified personnel w/out sacrifice of quality Useful as training tool for maintenance & use Message: development of EM system a science, not art – – – – Each process describable Validity of approach defensible Realistic expectations of effort by whom and when Measurable project process 48

Thank You! (Questions?) (Comments?) 49