Phase 5: Evaluation

This step requires a clear understanding of the stated business goals. A key objective is to determine how well results answer your organization’s business goals and whether there is some important business issue that has not been sufficiently considered. At the end of this phase, a decision on the use of the data mining results should be reached.

Adapting this phase to address context changes and model reuse handling involves:

  • We need an enhanced task for assesing whether the versatile model meets the business objectives and criteria in all the relevant contexts where they are to be deployed.
  • We need to decide whether the versatile model is able to be reused and adapted to the deployment data or not.

Tasks:

  • Task: Unlike the previous evaluation steps which dealt with factors such as the accuracy and generality of the model, in this step we need to assess the degree to which the model meets the business objectives and, thus, this step requires a clear understanding of the stated business goals. We need to determine if there is some business reason why this model is deficient, if results are stated clearly, if there are novel or unique findings that should be highlighted, if results raised additional questions, etc.
  • Outputs:
    • Assessment of data mining results with respect to business success criteria: Summarize assessment results in terms of business success criteria, interpret the data mining results, check the impact of result for initial application goal in the project, see if the discovered information is novel and useful, rank the results, state conclusions, check whether results cover all contexts relevant for the business success criteria, etc.
    • Approved models: Select those (versatile) models which, after the previous assessment with respect to business success criteria, meet the the selected criteria.
  • Task: Extra time for reflection on the successes and weaknesses of the process just completed. Although the resulting models appear to be satisfactory and to satisfy business needs, it would be appropriate to do a more thorough review of the whole data mining process seeking for overlooked tasks and quality assurance issues. We should summarise activities and decisions made in each phase learning thus from your experience so that future data mining projects will be more effective.
  • Outputs:
    • Review of process: Summarize the process review and all the activities and decisions for each phase. Give hints for activities that have been missed and/or should be repeated.
  • Task: Depending on the results of the reviewing the process of the initial data mining project, the project team decides how to proceed. The team decides whether (a) to continue to the deployment phase, (b) go back and refine or replace your models thus initiating further iterations, or (c) set up new data mining projects. This task includes analyses of remaining resources and budget, which may influence the decisions. If the results satisfactorily meet your data mining and business goals, start the deployment phase.
  • Outputs:
    • List of possible actions: List possible further actions along with the reasons for and against each option: analyse potential for deployment and improvement(for each result obtained), recommend alternative following phases, refine the whole process, etc.
    • Decision: Describe the decision made: rank alternatives, document reasons for the choice and how to proceed along with the rationale.

Previous Phase <– Phase 4: Modelling   —   Next Phase –> Phase 6: Deployment


 

Legend of the different representation of original and new/enhanced tasks and outputs: