PBIS and You

Overview

What is the purpose of the Performance-Based Incentive System (PBIS)?
The goal of the Performance-Based Incentive System (PBIS) is to ensure good results for young children and their families through Smart Start in every county in North Carolina. PBIS provides a mechanism whereby local partnerships can be rewarded for excellent results and where local partnerships that are not achieving results will be assisted with more intensive technical assistance.

Why is this system being developed now?
The General Assembly has mandated that a system must be developed to measure the performance of local partnerships. Smart Start is now funded in all 100 counties and full funding is within reach. All local partnerships have had funding and enough time to begin the development of a system of early care and education. Now is the time to develop the system that will report results.

Is this system approved by the NC General Assembly?
The General Assembly mandated that a system should be developed by the North Carolina Partnership for Children to measure local partnership performance. The authorizing legislation follows.

143B-168.12(a) (7)
“The North Carolina Partnership may adjust its allocation on the basis of local partnerships’ performance assessments. In determining whether to adjust its allocations to local partnerships, the North Carolina Partnership shall consider whether the local partnerships are meeting the outcome goals and objectives of the North Carolina Partnership and the goals and objectives as set forth by the local partnerships in their approved annual program plans.

The North Carolina Partnership may use additional factors to determine whether to adjust the local partnerships’ allocations. These additional factors shall be developed with input from the local partnerships and shall be communicated to the local partnerships when the additional factors are selected. These additional factors may include board involvement, family and community outreach, collaboration among public and private agencies, and family involvement.

On the basis of performance assessments, local partnerships annually shall be rated ‘superior’, ‘satisfactory’, or ‘needs improvement’. Local partnerships rated ‘superior’ may receive, to the extent funds are available, a ten percent (10%) increase in their annual funding allocation. Local partnerships rated ‘needs improvement’ may receive up to ninety percent 90%) of their annual funding allocation.

The North Carolina Partnership may contract with outside firms to conduct the performance assessments of local partnerships.”

In response to this legislation, the North Carolina Partnership for Children developed the Performance-Based Incentive System. The “minimum” and “high-performing” standards equate to “satisfactory” and “superior”, respectively, in the legislation.

What are the guiding principles used in designing the PBIS?

• Local partnerships must be involved in designing the overall incentive system
• The system must be flexible
• The system should address equity issues
• The system should reward performance as well as impose sanctions
• The system must establish minimum standards of performance
• Data should be objectively collected, valid and reliable

Who was on the Design Team for the development of PBIS?
The Design Team was made up of Local Partnership Board Members, NCPC Board Members and Staff, Frank Porter Graham Researchers and Evaluators, Division of Child Development Staff, and National Experts and Researchers.

Individuals included:
Thabiti Anyabwile, former NCPC Training Director
Joyce Baffi, NCPC Deputy Director
Roger Bailey, Davidson Co. Partnership for Children Director
Peggy Ball, DCD Interim Director
Kathleen Bernier, FPG Evaluator and Researcher
Linda Blanton, Cumberland County
Sandy Brennaman, Consultant and Cleveland Board
Donna Bryant, FPG Evaluator and Researcher
Dick Clifford, FPG Researcher
Gale Cruise, Stokes Partnership ED
Angela Deal, Burke Partnership ED
James Dodson, NCPC IT Director
Jana Fleming, FPG Evaluator and Researcher
Cheryl Hayes, Finance Project Director, Washington DC
Connie Holton, NCPC Finance Director
Erica Holton, NCPC Contracts Specialist
Olson Huff, Pediatrician and NCPC Board
Judy Langford, Consultant and Facilitator
Henry Lister, Wake Co. Partnership for Children Evaluator
Schatzi McCarthy, NCPC Planning Manager
Jenny Megginson, Chatham Partnership ED
Duncan Munn, DHHS- Developmental Disabilities Director
Marsha Munn, NCPC Program Manager
Ron Penney, DCD Finance Director
Karen Ponder, NCPC ED
Michele Rivest, Orange Partnership ED
Sue Russell, Child Care Services Association Director and NCPC Board
John Shively, Kansas City
Tony Solari, NCPC Policy Analyst
Julie Tunney, Beaufort Partnership ED
Joy Sotolongo, FPG Evaluator
Dr. Abe Wandersman, University of South Carolina
Chris VanNess, Kansas City
Sara Watson, Finance Project Consultant, Washington DC

How and when will this be implemented?
The implementation plan is not yet complete but once it begins, it will include a one-year hold harmless period in which the criteria and standards will be field tested and necessary adjustments made.

How will the PBIS be funded?
The funding will be addressed in the full implementation plan. Various options are currently being studied.

After discussion of performance criteria, what are the next steps?
Once the Design Team receives feedback on the criteria, appropriate adjustments will be made and the Design Team’s recommendation will go to the NCPC Board for approval. Following approval by the board and the gathering of necessary data for evaluation, the implementation will begin with a one year hold harmless period.

If I disagree with some of the measurements and criteria, can or will they be changed?
The PBIS Design Team will consider all comments and suggestions that are received in writing by the deadline. Appropriate changes will be made before they are approved by the NCPC Board.

What happens if partnerships do not achieve a minimum level of results?
Since the overall goal of PBIS is to achieve results for children, partnerships that do not achieve a minimum level of results will be provided assistance to improve their performance. The assistance will be progressive beginning with very focused technical assistance. If no progress is made, a formal corrective action plan will be put in place. If no progress is made after this plan is put in place, funding may be decreased as outlined in the legislation. The final response to no results will be a temporary transfer of the administrative responsibilities to the North Carolina Partnership for Children.

How will the actual values for these indicators be reported to the public?
A report of the results must be given to the General Assembly on an annual basis and consequently is available to the general public.

How will data be collected to ensure consistency and equity in evaluation across LPs?
Statewide databases will be used as well as standardization of the survey instruments.

How should local partnerships involved in strategic planning use these proposed criteria?
All of the criteria are based on the Smart Start core services and the performance standards. Therefore there should be no inconsistency in a local partnership’s strategic plan and the PBIS criteria.

How will the new statewide school readiness measurement system be connected to the PBIS?
There should be a direct connection since everything we’re doing through Smart Start should be preparing children for school success. PBIS will measure our success in achieving system results that lead to better school readiness. If children are then assessed and found not to be ready for success in school, local partnerships and the community will need to re-examine their services and levels of collaboration to see what else is needed or what needs to be changed.

Are these true standards of excellence or will these standards be raised overtime?
These standards were created in the Fall of 2000, looking at where North Carolina’s children, families and child care providers were, examining the program goals and setting achievable high standards. These standards of excellence and minimum standards will be re-evaluated every two years and re-established until we have reached all of the NCPC program goals.

How will ongoing research and evaluation efforts in and of Smart Start affect the minimum standards and standards of excellence?
As part of our efforts for continuing quality improvement, NCPC is committed to on-going research, evaluation and development to inform all of our practices. Possible outcomes include refined measures, new measures, new data sources and collection measures and substantiation for promising practices.

Why did Smart Start wait 10 years to create specific standards for partnerships?
Smart Start services were not available statewide until 1998. Development of these standards began in 1999 and involved input from myriad resources and experts in North Carolina and nationally. Data collection takes time and 2002 served as the base year for most standards. Now that most Smart Start partnerships have quality services in place, this system will help ensure Smart Start resources are getting the results we know prepare children for school and ensure communities are supporting the needs of young children.

How can Smart Start Partnerships be held accountable for standards that are out of their control?
Smart Start partnerships are the community-level leaders in establishing a comprehensive early education system where every child has the services they need to succeed. While Smart Start partnerships are not solely responsible for providing those services, they are charged with identifying where gaps in services exist and how best to address the needs. Partnerships are held accountable for meeting the minimum standards that are need to make up an effective early childhood system.

How have budget cuts affected Partnership’s ability to meet these standards?
There is no doubt that budget cuts have affected Smart Start’s ability to deliver services to all the children who need them. However, in spite of the cuts, progress continues to be made. Even further progress could have been achieved with additional funding.

How do these standards ensure children’s readiness for school?
Research tells us the components that are needed to ensure school readiness, such as teacher education and high quality child care. These standards focus on making sure every county builds an early childhood system that incorporates the proven components to school readiness.

Do these standards fundamentally change the Smart Start program?
No. This system provides a measurable, consistent way to judge progress toward creating communities that adequately support young children and families. Smart Start’s goal has always been to ensure children are healthy and ready to succeed. This is just another way to ensure we’re making progress and to provide technical assistance and support where it is most needed.

What if your county doesn’t make progress or reach these goals?
The North Carolina Partnership for Children will help partnerships devise a plan to reach each goal. Progress will be monitored closely and support and technical assistance will be integral to achieving the desired results.

Who gathers this data/information?
More than 20 credible, independent sources are used to collect and analyze the data. All data is collected through state databases.

Administration

Aren’t the audit results too old to be useful?
We realize that using audit results may not be ideal. However, these reports are the best information available in that the results are standardized and consistent for all partnerships. Furthermore, the audit reports carry weight in the eyes of the general public because they are performed by an independent outside agency. As a result, these audit results are most closely identified in the eyes of the public with the local partnerships' ability to effectively manage the financial aspects of the Smart Start program.

Family Support

Many partnerships and their service providers already do satisfaction telephone or mail surveys. How will these efforts be coordinated so as not to burden families?
The field tests in November will provide the opportunity for NCPC and local partnerships to learn how to coordinate survey strategies that can minimize or eliminate redundant questioning. The indicators proposed for PBIS are few in number and ultimately could be collected in the course of more in-depth regular program evaluation.

Practical lessons on this issue from five different partnerships will be incorporated into the implementation plans for PBIS. These plans will include systematic methods for developing and implementing this requirement in each partnership, fitting their respective program and evaluation needs and capabilities.

The Family Support Committee of the PBIS project is now working with staff from the Division of Social Services to integrate surveys and other evaluation efforts aimed at family support. We anticipate that this effort can also cut down on duplication of family surveys, improve comprehensiveness in reporting for programs with multiple funding sources, and focus on needed joint evaluation projects.

In the area of family support, why require uniform general indicators of family perceptions and capabilities even though county and family circumstances and needs as well as service programs vary so much?
We have carefully chosen a few core measures that simply represent key aspects of families’ needs and experiences with family support activities. These indicators are consistent with commonly accepted family support principles and are intended to give a useful and relevant snapshot of this complex area.

We are developing further plans for systematic research and demonstration to refine our knowledge of measuring family support in concert with states and research organizations from around the nation. Our plans are to develop measures that are useful both at the family and service level as well as for public consumption. In-depth evaluation will always be required to “prove” the effectiveness of individual programs—we are attempting to build a framework for such efforts as well as to connect them to the well-being of children within communities.

Why should local partnerships be measured on the extent to which documented needs match up to the percentages of Smart Start funds allocated to different areas, e.g., family support?
Local partnerships are accountable for how their budget priorities address local needs. Although the Smart Start funding cannot support every community effort aimed at particular issues such as family stability or violence, these funds are important investments. The local partnership boards’ decisions can leverage other funds for such priorities as well as foster partnerships to address common needs of the community.

This indicator is to highlight the thought process and documentation related to priorities, not to prescribe particular target issues or a numerical formula. This minimum requirement simply asks that local partnerships examine the well being of families using available data and to relate this evidence to their decisions about program priorities. This indicator also includes the specific measures identified in the Health section.

The incentive opportunity for the partnership in this area is to adopt goals for which progress will be measured over time with countywide measures. Smart Start partnerships that accept the challenge to work for measurable community change in areas such as violence, literacy, abuse, living wages, etc., will be rewarded for success when funds are available.

In the Family Support area, how will local partnerships and direct service providers be able to follow up with families regarding concerns expressed in the post card survey?
Responses will be tabulated for each program and reported in summary form to each partnership. Where there are specific concerns that warrant special attention and enough information to direct a response, they will be shared with the partnership office immediately.

A short statement about the purpose of the post card highlights its use to describe the status of families rather than conduct an in-depth evaluation of the family support program. However, the post cards will have an appropriate telephone number and encourage respondents to call for further information or assistance in problem solving related to their services.

Families surveyed will be given a summary report of the findings and local partnerships and service contractors will be encouraged to tell families what the results mean to them.

How can partnerships be accountable for changes in the lives of families or health status if most of our money must be spent in child care?
The mission of a local partnership is to design a system of care and education for children 0-5. This plan should be comprehensive and address health needs as well as family support needs related to young children. Smart Start funding primarily addresses the needs of children related to child care but Smart Start funding is only a part of the funding that is needed to meet all the needs of children. Local Partnership Boards are charged with bringing everyone to the table to solve the problems and much more money is available collectively to affect all the needs. Local Boards must be deliberate in identifying the needs and seeking additional funds to meet them.

Will local partnerships be held accountable for health and family support standards if they choose not to spend Smart Start dollars to one or both of these areas?
Yes, local partnerships’ were created to serve as the coordinators and planners of services for young children. Smart Start dollars are only a part of funding that helps to meet the needs, including health and family support.