Evaluation of the Community Futures Program in Quebec – Final Report

This report presents the outcomes of the evaluation of the Community Futures Program (CFP) administered in Quebec by Canada Economic Development.

The focus of the evaluation begins in April 1995 and ends on March 31, 2007.

The purpose of this review is to evaluate the program’s relevance, design, delivery and outcomes and to evaluate cost-effectiveness and alternatives.

Table of Contents

  1. Executive Summary
  2. Introduction
    1. Description of the CFP
      1. National context
      2. Program rationale, mandate and objectives
      3. Quebec context
  3. Evaluation Methodology
    1. Evaluation questions
    2. Data collection methods
      1. Document review and literature review
      2. Stakeholder interviews
      3. Surveys
      4. Secondary data
      5. Case studies
    3. Limitations of the methodology
    4. Limitations of the evaluation period
  4. Analysis
    1. Relevance
      1. Is there a continued need for the CFP? To what extent does this need vary from one RCM to another?
      2. Does the CFP overlap or complement other government programs or other private sector services?
      3. Are the objectives and activities of CFP-funded organizations in line with national CFP objectives? Are they in line with the Agency’s objectives?
    2. Design and delivery
      1. Are the national and provincial networks effective? Is the provincial network in line with Agency priorities?
      2. What factors facilitate or impact the achievement of program outcomes?
      3. Is the resource allocation model used by the Agency adapted to the region’s needs?
      4. To what extent are the organizations’ strategic planning and the community’s development plans developed by the community? Are the organizations’ activities in line with these plans?
      5. Are the Agency and organizations gathering the necessary data to measure program performance and conduct future evaluations? What points of comparison are used to evaluate the outcomes?
      6. Are the investment funds well managed? Are the number of loans, the loan amounts and the risk level adapted to needs?
    3. Program impact – short term
      1. To what extent has the CFP provided appropriate information, referrals and advice to clients?
      2. To what extent has the CFP improved clients’ business knowledge and skills?
      3. To what extent has the CFP created new businesses or strengthened existing businesses?
      4. To what extent does the CFP meet the needs of target groups? Are the right groups targeted?
    4. Program impact – intermediate
      1. To what extent has the CFP stimulated communities’ economic development, helped develop and diversify their economy, and helped strengthen their capacity for action?
    5. Program impact – long term
      1. To what extent has the CFP helped achieve long-term goals (i.e., economic growth and stability, diversification and development of rural communities, sustainable communities, and survival of business assisted by local organizations)?
    6. Cost-effectiveness and alternatives
      1. To what extent is the CFP cost-effective?
      2. Are there other more cost-effective/ efficient approaches or alternatives that would achieve CFP objectives?
    7. Recommendations
    8. Suggested improvements
  5. List of tables
  6. List of figures
  7. List of acronyms
  8. Recommendations
  9. Appendix A CFP Logic Model
  10. Appendix B List of Community Futures Organizations
  11. Appendix C Evaluation matrix
  12. Appendix D List of reviewed documents
  13. Appendix E Interview guides
  14. Appendix F (a) CFDC/BDC/CEDC survey questionnaire
  15. Appendix F (b) CFDC/BDC/CEDC survey frequency tables
  16. Appendix G (a) CFDC and BDC client survey questionnaire
  17. Appendix G (b) CFDC and BDC client survey frequency tables
  18. Appendix H Administrative report of the survey of CFDC and BDC clients
  19. Appendix I Case studies
  20. Appendix J Detailed performance measurements in the agreements between the Agency and the CFDC Network