Loading...
8 results
Search Results
Now showing 1 - 8 of 8
- Analysis of complementary methodologies for the estimation of school value addedPublication . Portela, M. C. A. S.; Camanho, A. S.This paper analyses the value added (VA) of a sample of Portuguese schools using two methodologies: data envelopment analysis (DEA) and the methodology used presently by the UK Department for Children, Schools and Families (DCSF). The VA estimates obtained by the two methods are substantially different. This reflects their different focus: DEA emphasizes on best-observed performance, whereas the DCSF method reveals average performance. The main advantage of the methodology used by the DCSF is its simplicity, although it confounds pupil effects with school effects in the estimation of school VA. In contrast, the DEA methodology can differentiate these effects, but the complexity may prevent its use in a systematic way. This paper shows that the two methods provide complementary information regarding the VA of schools, and their joint use can improve the understanding of the relative effectiveness of schools regarding the progress that pupils make between educational stages.
- An improvement on the Tracy and Chen model 'A generalized model for weight restrictions in DEA'Publication . Khalili, M.; Camanho, A. S.; Portela, M. C. A. S.; Alirezaee, M. R.Recently Tracy and Chen presented a parametric DEA model (PDEA) to assess relative efficiency in the presence of a generalized form of linear weight restrictions. This paper proposes a modification to the PDEA model that avoids the need to resort to searching algorithms to estimate efficiency, and assures that the correct efficiency scores are obtained in a single stage using mathematical programming solvers. The results of this model and the results of Tracy and Chen's PDEA model are compared using the examples reported in their paper. The results confirm the superiority of the model proposed in this paper.
- Benchmarking clinical practice in surgery: looking beyond traditional mortality ratesPublication . Castro, Ricardo A. S.; Oliveira, Pedro N.; Portela, Maria; Camanho, Ana S.; Queiroz e Melo, JoãoThis paper proposes two new measures to assess performance of surgical practice based on observed mortality: reliability, measured as the area under the ROC curve and a living score, the sum of individual risk among surviving patients, divided by the total number of patients. A Monte Carlo simulation of surgeons' practice was used for conceptual validation and an analysis of a real-world hospital department was used for managerial validation. We modelled surgical practice as a bivariate distribution function of risk and final state. We sampled 250 distributions, varying the maximum risk each surgeon faced, the distribution of risk among dead patients, the mortality rate and the number of surgeries performed yearly. We applied the measures developed to a Portuguese cardiothoracic department. We found that the joint use of the reliability and living score measures overcomes the limitations of risk adjustedmortality rates, as it enables a different valuation of deaths, according to their risk levels. Reliability favours surgeons with casualties, predominantly, in high values of risk and penalizes surgeons with deaths in relatively low levels of risk. The living score is positively influenced by the maximum risk for which a surgeon yields surviving patients. These measures enable a deeper understanding of surgical practice and, as risk adjusted mortality rates, they rely only on mortality and risk scores data. The case study revealed that the performance of the department analysed could be improved with enhanced policies of risk management, involving the assignment of surgeries based on surgeon's reliability and living score.
- Performance assessment of secondary schools: the snapshot of a country taken by DEAPublication . Portela, M. C. S.; Camanho, A. S.; Borges, D.This paper describes a performance assessment of Portuguese secondary schools using data envelopment analysis (DEA). The assessment adopts a perspective where schools are viewed as promoting students achievement given their characteristics in terms of academic abilities and socio-economic background. Our sample comprised all secondary schools in Portugal with both basic and secondary education levels. Two types of DEA analysis are performed: one using an output-oriented model that restricts output (exam scores) weights to be linked to the number of students that have done that exam in the school, and the other using a model that restricts factor weights to be equal for all schools. In this model the weight restrictions are linked to the total number of exams done nationally. The first model is well suited for identifying worst performing schools and to assess schools that may specialize in certain subjects, whereas the latter is best suited for improving discrimination between best performing schools when pursuing the identification of benchmarks, as well as to construct performance rankings.
- BESP – benchmarking of Portuguese secondary schoolsPublication . Portela, Maria; Camanho, A. S.; Borges, Diogo NóvoaPurpose – The purpose of this paper is to illustrate the implementation of a web-based platform integrating benchmarking and data envelopment analysis (DEA) for the Portuguese secondary schools. Design/methodology/approach – The benchmarking platform is designed around a set of key performance indicators that are displayed using benchmarking graphs. These indicators are also aggregated through the methodology of DEA to provide a summary measure of performance. Findings – The benchmarking platform developed enables schools to perform internal and external evaluation through a standard model that is based on indicators of school activities. It encourages schools’ efforts of continuous improvement and increases society awareness regarding schools’ context and results obtained. Practical implications – The benchmarking platform can be useful for schools and general public. For the general public, there is a tool that allows the construction of user-defined rankings online and benchmarking tools that allow the comparison of performance of a specific school with others. In addition to these features, schools have in the BESP platform a repository of historical data, and the possibility to see a set of graphs that show for some indicators their evolution over time. Originality/value – This paper describes a breakthrough in the Portuguese education context. The BESP platform is the first in this context to combine DEA and benchmarking tools in a web-based environment, designed to enable real-time performance assessments.
- Benchmarking of secondary schools based on students’ results in higher educationPublication . Silva, Maria C. A.; Camanho, Ana S.; Barbosa, FláviaThe performance of secondary schools is usually assessed based on students’ results on national exams at the end of secondary education. This research uses data on academic achievements by first-year univer- sity students to benchmark secondary schools on their ability to lead students to success in higher edu- cation. The analysis is conducted using data of University of Porto and Catholic University of Porto, Portu- gal, for a three-year period, corresponding to more than 10.0 0 0 students from 65 degrees, for which the school of origin is known. A number of variables representing students’ success in Higher education were constructed for each school in our sample and aggregated through a Benefit of the Doubt indicator. Re- sults suggest that the schools’ ranking based on schools’ ability to prepare students for university success is quite different from the ranking based on results on national exams. Given these findings, we propose complementing schools’ performance assessments (traditionally based on national exam results or indi- cators of value added) with indicators that account for the preparation of students for success in future challenges, which is indisputably a key objective of secondary education. We propose a composite indi- cator for the analysis of these complementary aims as well, and results show that frontier units indeed exhibit trade offs between traditional measures of performance and our new measure of performance.
- Assessing the evolution of school performance and value-added: trends over four yearsPublication . Portela, M. C.; Camanho, A. S.; Keshvari, A.This paper explores the changes in value added (VA) of a sample of schools for cohorts of students finishing secondary education between 2005 and 2008. VA estimates are based on distance measures obtained from DEA models. These measures are computed for each pupil in each school, and evaluate the distance between the school frontier in a given year and a pooled frontier comprising all schools analysed. The school VA is then computed by aggregating the VA scores for the cohort of pupils attending that school in a given year. The ratio between VA estimates for two consecutive cohorts, that attended the school in different years, is taken as the index of VA change. However, the evolution of school performance over time should consider not only the movements of the school frontier, but should also take into account other effects, such as the proximity of the students to the best-practices, represented by the school frontier, observed over time. For that purpose we developed an enhanced Malmquist index to evaluate the evolution of school performance over time. One of the components of the Malmquist index proposed measures VA change, and the other measures the ability of all school students to move closer to their own school best practices over time. The approach developed is applied to a sample of Portuguese secondary schools.
- Benchmarking Hospitals through a web based platformPublication . Portela, Maria; Camanho, A. S.; Almeida, Diogo Queiroz; Nogueira, Luís; Silva, Sofia Nogueira da; Castro, Ricardo A. S.Purpose – In a context of international economic crisis the improvement in the efficiency and productivity of public services is seen as a way to maintain high-quality levels at lower costs. Increased productivity can be promoted through benchmarking exercises, where key performance indicators (KPIs), individually or aggregated, are used to compare health units. The purpose of this paper is to describe a benchmarking platform, called Hospital Benchmarking (HOBE), where hospital’s services are used as the unit of analysis. Design/methodology/approach – HOBE platform includes a set of managerial indicators through which hospital services’ are compared. The platform also benchmarks services through aggregate service indicators, and provides an aggregate measure of hospital’s performance based on a composite indicator of the service’s performances. These aggregate indicators were obtained through data envelopment analysis (DEA). Findings – Some results are presented for Portuguese hospitals for the trial years of 2008 and 2009, for which data is publicly available. Details for the service-level analysis are provided for a sample hospital, as well as details on the aggregate performance resulting from services performances. Practical implications – HOBE’s features and outcomes show that the platform can be used to guide management actions and to support the design of health policies by administrative authorities, provided that good quality and timely data are available, and that hospitals are involved in the design of the KPIs. Originality/value – The platform is innovative in the sense that it bases its analysis on hospital’s services, which are in general more comparable among hospitals than indicators of hospital overall performance. In addition, it makes use of DEA to aggregate performance indicators, allowing for user choice in the inputs and outputs to be aggregated, and it proposes a novel model to aggregate service’s efficiencies into a single measure of hospital performance.