1

Credit for Low-Income Students and Access to Higher Education in Colombia:

A Regression Discontinuity Approach

Tatiana Melguizo* Rossier School of Education

University of Southern California

Fabio Jose Sanchez Juliana Marquez

Los Andes University

This study was supported by a small grant by the Spencer Foundation (2011-00128). Opinions are those of the authors alone and do not necessarily reflect those of the granting agencies or of the authors’ home institutions. Acknowledgements: We would like to thank Juan Esteban Saavedra and Judy Scott-Clayton for comments on earlier versions of the paper. *Rossier School of Education, 3470 Trousdale Parkway, WPH 702 G, Los Angeles, CA, 90089. Phone: (213) 740 3635 Fax: (213) 740 3889 Email: melguizo@usc.edu.

2

Abstract

This study evaluates the impact of a national level loan program on the college enrollment rates of low-income students. We use national level data along with a discrete time survival model embedded in a regression discontinuity design (RDD) to estimate the intent to treat of the program for the students at the margin. The results of the study confirm that the program was effective in terms of increasing the potential number of low-income students who would have enrolled in college in the absence of the program.

Despite a substantial increase in college enrollment rates in Colombia over the

last decade, there are still wide disparities in access by socioeconomic status (Gaviria &

Toro, 2012; World Bank, 2012). According to the World Bank one of the main reasons

for the disparities may be the lack of a well-developed financial aid system (World Bank,

2003). As a result, the Colombian government has made an effort to increase the financial

aid available to low- and middle-income students in the country. In 2002, with support

from a World Bank loan the government designed and funded the program Acceso con

Calidad a la Educación Superior (Access with Quality to Higher Education, or ACCES)

to expand the opportunity to enter higher education to low- and middle-income students.

The main objective of the present study is to estimate the effect of the ACCES

loan plus grant program, on the enrollment of low-income students to a postsecondary

institution in Colombia. Thus, the research question that guides this study is: has the

ACCES loan plus grant program been effective in increasing the likelihood of college

enrollment for eligible low-income students? We use a discrete-time survival model

(Singer & Willett, 1993) to estimate the risk of enrollment within an RDD setting (Lesik,

2006). We are limited to providing only intent to treat (ITT) estimates, given that we

have three types of eligible students who did not comply with the treatment: 1) not all

eligible students applied to the program, 2) according to the funding agency the average

take up rate was about 70 percent, and 3) some of the students who applied were already

enrolled in college.1 As part of our RDD strategy we restricted the sample to students

1 The students who were enrolled in college at the time that they applied for the ACCES program may have been aware they were eligible for the program at the time that they enrolled in college.

3

who were either 5 percent above (eligible) or 5 percent below (non-eligible) according to

the academic performance criteria (i.e., score on SABER 11 the high school exit exam)

set for eligibility by the Colombian Institute for Educational Loans and Studies Abroad

(ICETEX).2 In addition, the program was designed to use different cut scores for students

in different geographic regions of the country, providing a solid identification strategy

with strong internal validity. This rigorous methodological strategy coupled with access

to complete national data suggest that the results of the study have both strong internal

and external validity. The results of the study confirm that the program was effective in

increasing the potential number of low-income students who would have enrolled in

college in the absence of the program.

This study contributes to the literature by estimating the effect of a national level

loan program that has a subsidy for low-income students. As described in detail below,

there is very little evidence regarding the impact of financial aid programs in the form of

loans (Chen & Desjardins, 2010; Dynarski, 2002, 2005; Long, 2004; Singell, 2004). A

number of descriptive studies in Colombia suggest a positive association between

receiving financial aid and college access and persistence (Cerdán-Infantes & Blom,

2007; Author, 2011). However, these estimates might be biased given that application to

financial aid programs is voluntary, and one would expect more academically prepared

and motivated students to apply to these programs (Dynarski, 2002). Our study improves

on the previous literature, using an appropriate estimation strategy that reduces the threat

of having biased estimates, and in addition providing information on the timing of

enrollment of the student population that was the focus of the policy.

The following is the structure of the paper. We begin by briefly describing the

structure of the higher education system in Colombia. This is followed by a review of the

financial aid literature both in the U.S. and Colombia. We describe the selection process

into the ACCES program as well as the proposed methodological strategy. The results are

reported, followed by a summary of the main conclusions and policy implications.

Characteristics of the Colombian Postsecondary Education System

2 ICETEX, the acronym in Spanish stands for: Instituto Colombiano de Crédito Educativo y Estudios en el Exterior.

4

Colombia’s postsecondary education system expanded substantially in the early

1990s, resulting in an over-representation of private postsecondary institutions (about 70

percent).3 The system is comprised of about 305 postsecondary institutions, the majority

of which are either universities or university institutions (193), the rest technical and

technical-professional institutes. There has been a substantial increase in access to

postsecondary education in the last decade (Author, 2012a). The percentage of students

enrolled increased from about 24.4 percent (the Latin American average) in 2002 to about

37.1 percent in 2010 (Gaviria & Toro, 2012; World Bank, 2012). Despite having

enrollment rates above the regional average, there are wide disparities in access to

postsecondary education by socioeconomic status. Colombia has had two big expansions

of its higher education system in the last two decades. Before the first expansion of the

system in the early 1990s, approximately two percent of the relevant age-cohort

population was enrolled in tertiary education, compared to 20 percent for the best-off

quintile. After the expansion, the largest gains in coverage occurred in the best-off

quintile, where net coverage rose from 23 percent to 40 percent. However, it is worth

noting that there was also an expansion of about 170 percent in the first quintile (World

Bank, 2003). Colombia had a second large expansion in college enrollment in the last

decade. Recent evidence suggest that this time enrollment rates substantially increased

the proportion of students from the lowest incomes (Author, 2012a; World Bank, 2012).

The percentage of low-income students (from families with less than two minimum wage

incomes per month (proxy for socioeconomic status)) increased from 26.64 percent to

over 40 percent between 2000 and 2009. Unfortunately, the increase in enrollment for

this group of students mostly happened at the technical and technological institutions that

traditionally have fewer resources to support students and hence exhibit high dropout

rates (about 60 percent) (Author, 2012a). Glaring socioeconomic inequalities have been

a constant since the inception of the higher education system in Colombia. In the mid

1980s Jimenez and Tang (1987) did a survey of high school seniors in Colombia and

found that the socio-economic distribution of students in postsecondary institutions

would substantially change if selection was based purely on merit. Most recently Gaviria

3 For a more detailed description of the tertiary education system in Colombia see World Bank (2012) and World Bank (2003).

5

and Toro (2012) also found evidence that the private selective institutions are mostly

enrolling students from the highest socioeconomic status of the country. In summary, the

country has currently an enrollment rate above the Latin American region and despite

recent gains in access for students from low-income households; substantial inequalities

in terms of access by level of income and type of institution (i.e., elite private) still

remain.

Enrollment and Drop Out Rates

Despite a substantial increase in the enrollment rates of students in the system, the

dropout rates are high and even increased during the years of the expansion. The dropout

rates have risen substantially in the last decade, the five-year dropout rate for the 2000

students’ cohort was 50.9 percent and it increased to 55.1 percent for the 2005 cohort.

There are substantial differences in dropout rates by type of postsecondary institution.

The first-year dropout rates of students enrolled at universities in 2009 were around 20

percent compared to 24 and 39 percent for students enrolled in technical and

technological institutions (Author, 2012a). The Ministry of Education (MEN) in

collaboration with the University of Los Andes, developed a system to collect and

analyze data related to college dropout and persistence, el Sistema de Prevención y

Análisis de la Deserción en Educación Superior, SPADIES. This system has collected

information about enrollment and dropout for every student in every postsecondary

institution in the country since 1998. According to the SPADIES the first semester

dropout rates have been increasing steadily in the last decade. Specifically, the rate

increased from about 18 to 24 percent between 2000 and 2009 (Author, 2012a). The

explanation for the increase in the dropout rates is related to the expansion of the system

during the same period. As mentioned above, the gross enrollment rate increased from 24

to more 30 percent from 1998 to 2008 (above the regional mean), and most of the access

growth came from the students of the lowest income levels (between 1 and 2 minimum

wage of monthly income), and from students with relatively low scores on the national

high school exit exam (SABER 11), undertaken by the Colombian Institute for the

Promotion of Postsecondary Education (ICFES) (MEN-CEDE, 2009; Author, 2012a).

The SPADIES also provides detailed information on dropout rates by semester

and by different types of individual (i.e. socioeconomic status and SABER 11 scores),

6

and institutional (i.e. type and control) characteristics. Analysis of SPADIES data for the

period analyzed suggest that there is wide variation in first year dropout rates by level of

academic abilities measured by the SABER 11 score (i.e. low, medium, and high). The

first year dropout rate for students with high SABER 11 scores is 15 percent compared to

30 percent for those with the lowest scores. The SPADIES can be used to make

comparisons for students in the low- and high-income groups and low and high SABER

11 scores. The evidence confirms that dropout rates increase for students in the low

income and low score groups. The highest first year dropout rates were for students in the

lowest income group and with the lower scores, 35 percent, compared to 15 percent for

the students in the highest income and highest score group. The dropout rates for the low

income and high score group were 20 percent, five percentage points higher than the one

for the highest income.

The previous results suggest that despite the government’s success in increasing

access to low-income students, the dropout rates have increased dramatically over the last

decade and in particular for low-income students (Author, 2012a).

Theoretical Framework and Review of the Literature

Becker’s (1967) human capital model predicts that individuals invest in education

until the present value of benefits equals costs. Individuals who have no access to

government assistance must rely either on family resources or credit to cover the costs.

Individuals who need to rely on lenders would need to provide assets as collateral. The

resulting constraint on borrowing implies that educational outcomes will be determined

not only by the costs and benefits of the investment, but also by preexisting inequalities

in family resources (Dynarski, 2002, 2003; Long, 2004).

Despite the clear theoretical predictions, there is relatively little empirical

evidence on the impact of financial aid programs in the form of subsidized and un-

subsidized loans (Singell, 2004; Chen & DesJardins, 2010). The primary barrier to

rigorous research in this area is the difficulty in identifying exogenous variation in

student loan access. Typically, loan access is correlated with a number of other factors

that may also affect college outcomes, including financial need, college selectivity, and

unobserved student characteristics like motivation. For this reason, simple comparisons

of students with and without loans or strategies that use matching techniques are prone to

7

bias. Only a handful of studies have been able to conduct a rigorous evaluation of

financial aid on different types of outcomes.

Financial aid and student’s college enrollment and persistence/dropout rates

A substantial body of literature explores the association between receiving

financial aid and college enrollment, persistence, and attainment (Cabrera, Stampen, &

Hansen, 1990; DesJardins, Ahlburg. & McCall, 2006; Dynarski, 2002, 2003; Hansen,

1983; Heller, 1997; Jackson, 1988; Kane, 1994; Long, 2004; McPherson & Schapiro,

1991; St. John, 1990; St. John & Noell, 1989; Stinebrickner & Stinebrickner, 2008).

Heller (1997) reviewed around twenty studies published from the mid 1980s to the mid

1990s that explored the impact of financial aid on student’s enrollment. With the

exception of two studies (Hansen, 1983; Kane, 1994), most of the studies reviewed found

that decreases in financial aid led to declines in enrollment (Jackson, 1988; McPherson &

Schapiro, 1991; St. John, 1990; St. John & Noell, 1989). Heller (1997) concluded that in

most of the studies reviewed financial aid was positively associated with enrollment and

that low-income and minority students were more sensitive to changes in financial aid

than their white peers.

A limitation of the majority of the studies reviewed by Heller (1997) is that they

did not acknowledge the problem of simultaneity between aid eligibility and college

enrollment, persistence, and attainment. Dynarski (2003) argues that aid eligibility is

correlated with many observed and unobserved characteristics that affect college

decisions. In her study she used a change in aid policy in the early 1980s to identify the

effect of financial aid on college attendance and completion. She performed a difference-

in-difference (DID) estimation strategy and concluded that the elimination of the aid

program reduced college attendance probability by more than one third. The estimates

showed that an offer of $1,000 in grant aid increased the probability of attending college

by about 3.6 percentage points. She concluded that aid eligibility also appears to increase

school completion. The few rigorous evaluations of nationwide changes in financial aid

programs do not provide conclusive evidence. Our study will contribute to the literature

by conducting a rigorous evaluation of a country-wide credit program aimed at increasing

access and persistence of low- and middle-income students.

Description of the financial aid program-ACCES

8

One of the factors that has allegedly contributed to the relatively low enrollment

and high dropout rates of low-income students in Colombia is the lack of access to

financial aid. Before 2002 the government could only offer financial aid (either loans or

grants) to one in ten enrolled students, and access to student loans for low income

students in the country was extremely limited because in order to secure a loan, students

or their families had to provide real estate or other assets as collateral (Cerdan-Infantes &

Blom, 2007). To address these inequalities, in 2002 the government, with support from a

$200 million dollar loan from the World Bank plus $87 million that the country

committed to the program as part of the requirements of loan, created the ACCES

program, with the goal of increasing access while maintaining quality. The main

objectives of ACCES were to increase: 1) the number of professionals and technicians in

the country; 2) technical education enrollment, which currently represents only 12

percent of the total; 3) the quality of the educational programs and; 4) the access to loans

given that at that point only 12 percent of the enrolled students had access to loans from

the main public lender ICETEX.

The ACCES program, removes collateral requirements and simplifies application

procedures. ACCES primarily targets low-income students. An eligibility “score” is

calculated as a complex function primarily based on SABER 11 test scores,

socioeconomic status4, institutional quality and year of study (See Table A1). Eligibility

cutoffs are then established by “department” (geographic regions equivalent to U.S.

states), depending upon resource constraints and other factors such as socioeconomic

characteristics of the population. This unique feature of the program with multiple and

diverse cutoff points across the country brings the exogenous variation necessary to

properly identify the statistical models and get unbiased estimates of the program (Figure

1). Finally, students who did not meet the departmental cutoff may still be approved if

they are of low socioeconomic status defined as a SISBEN 1 or 2 beneficiaries, are

officially displaced by internal conflict, or meet other requirements to gain “exceptional”

status. The ACCES program combines a mix of grants (25 percent) and subsidized loans

4 Social strata and System of Selection of Beneficiaries for Social Programs (SISBEN) levels 1 or 2. SISBEN is a proxy means-test that combines assets, household characteristics and socio demographic information to create a continuous score – the SISBEN score. The SISBEN level is determined based on the score. For more information on the SISBEN please check the description of the program at the Department of National Planning webpage: http://www.sisben.gov.co/

9

(75 percent) for low-income students and loans for middle-income students. Eligible

applicants can receive a loan for up to 75 percent of tuition costs (subject to a program

maximum), with additional “living expenses” loans available for students attending

universities located in cities other than their place of residence. Upon completion,

students have a repayment grace period of one year after which they start repaying the

loan balance. Approximately 83% of loans are repaid (Shen & Ziderman, 2008).5

<<Figure 1>>

The ACCES program is administered by the ICETEX, a government agency

created over 60 years ago to provide loans to help students finance a number of different

undergraduate and graduate programs. While ICETEX has a number of different

programs, the one that is relevant for the study is ACCES, which offers long-term loans

to pursue an undergraduate degree in a public or private institution in Colombia. A

student with an ACCES loan receives the equivalent of up to 11 minimum wages per

semester. This covers the full tuition in public institutions and about 75 percent of the

cost in the most expensive private universities. The annual interest rate may change

during the time the student is enrolled and when the student graduates from college (for

the poorest students enrolled in universities, the interest rates were four percent while

enrolled and after graduation). Students from the two lowest income levels receive a

grant for 25 percent of the tuition.6 Students start to repay one month after the end of the

funded program of study. Most recently, it has been determined that ICETEX will

forgive100% of the loans to students from the poorest backgrounds who attained an

outstanding score on SABER PRO, the Colombian college exit exam (World Bank,

2012).

A recent World Bank evaluation of the program indicates that between 2002 and

2007 the percentage of ACCES recipients belonging to the lowest socioeconomic tiers

jumped from 30% to 69%. (Cerdán-Infantes & Blom, 2007). According to this study the

dropout rates were 30 percent lower for loan beneficiaries compared to non-beneficiaries

with the same observable characteristics. A recent study that tested the association

5 For a more detailed explanation of repayment options and interest rates see page 72 of World Bank, 2012. 6 For a more detailed explanation of the ACCES program in Spanish see https://www.icetex.gov.co/dnnpro5/es-co/créditoeducativo/estudiostécnicostecnológicosyuniversitarios/largoplazoacces.aspx

10

between different types of financial aid and drop out rates also found that drop out rates

for students who received any type of financial aid were 30 percent lower than students

with similar characteristics without any form of financial aid (Author, 2011). These

results suggest that student loans offer a cost-effective way to increase enrollment and

reduce dropout rates of students from low-income households. Nonetheless, these studies

did not address the problem of self-selection described above.

Our study uses a discrete-time survival model strategy within an RDD setting to

evaluate ACCES, the largest public financial aid program in Colombia. Our

methodological approach estimate timing of enrollment of eligible students between 2002

and 2010. The dataset, sample, and methodological strategy are described in detail below.

Data

Colombia is a world leader in developing high quality national-level datasets in

secondary and tertiary education. Our main data sources are the records of the SABER 11

–the compulsory high school exit exam - and the SPADIES. Together, these sources

allow us to determine whether and when a high school graduate enrolled in a

postsecondary institution. The SABER 11 data include academic, socioeconomic, and

demographic information of the students. SPADIES7, a database maintained by the

Colombian Education Ministry (MEN-Spanish acronym) includes information about

every single student enrolled in any institute of higher education since 1998. The

SPADIES database includes information from SABER 11 as well as information related

to the student’s academic performance in college and whether the student has been

beneficiary of any academic support, financial aid and loans including ACCES. ACCES

is awarded once the student has been accepted to an institution of higher education, given

compliance with credit requirements.

We focus on all the students who could have enrolled at college or university

from 2002 to 2010, namely, after ACCES implementation. Students could have applied

and received an ACCES loan plus grant after being enrolled in college. Our sample is

restricted to eligible students and our outcome of interest is likelihood or risk of enrolling

after high school completion. We limit our analysis to university – as opposed to

7 SPADIES is similar to U.S.’s National Student Clearinghouse (NSC). Like the NSC, the SPADIES coverage of institutions is not complete and some institutions have imperfect reporting.

11

technical or technological institute – potential applicants because demand exceeds loan

availability.

Methodology

Sample

The total sample includes information for 952,135 individuals who were eligible

to the program. We restricted the sample to individuals who were either 5 percent above

(eligible) or 5 percent below (non-eligible) the department’s cutoff, a total of 97,520 low-

income high school graduates who were ACCES eligible and who could have enrolled at

a college or university in Colombia between 2002 and 2010. As we are dealing with

longitudinal data on student enrollment over the course of 16 semesters,8 we assembled

the data as a person-period dataset, where each student has a separate row recording the

data for each semester (Singer & Willett, 1993; Lesik, 2006).

Estimation Strategy

Our proposed evaluation uses a discrete-time survival analyses (Singer & Willett,

1993) and embeds a regression discontinuity design (RDD) (Lesik, 2006; Murnane &

Willett, 2011) to estimate the intent to treat (ITT) of the program on the timing of

enrollment in postsecondary education of low-income students. The RDD technique is

strong in terms of internal validity of the estimates, but the findings only apply for the

students at the margin. In other words, our findings should be restricted to eligible low-

income students at the margin. No inferences can be made about students who had either

very high or very low scores on the high school exit exam. The study has a strong

external validity given that we have information for all the high school graduates in the

country between 2002 and 2010.

We estimate the ITT effect based on comparing students who were eligible to

ACCES with those who were not. Specifically, we applied the SABER 11 cutoff criteria

used in each department (state) and selected individuals who were five percentage points

above and five percentage points below the cut score in each department. Through a

discrete-time duration model we estimated the impact of being eligible for ACCES on the

risk or likelihood of transitioning from high school graduate to enrolling in college. The

8 By 2010, the 2002 high school graduates had been observed during 16 semesters.

12

demographic controls include gender, age, and mother’s highest level of education. We

use fixed effects by department to control for specific regional differences.

We specify a model where the individual student’s risk of enrolling in college is a

function of a number of individual characteristics, and a variable indicating eligibility.

Logit ENROLLh(ti) = β1ELIGi + ΣβjSEM-AFTER-HIGH SCHOOL GRADUATION +

Σ γiSEMi*ELIGi+ β17DISTi + β18DISTPosi+ β19DISTNegi+

β20DISTpossquarei+β21DISTnegsquarei +β22DEMi + β23FIXEDEFFECTDEPi + ui (1)

Measures

Outcome variable

h(ti) is the variable that indicates the hazard of enrolling in college in each of the

sixteen consecutive semesters, or eight years, after finishing high school between 2002

and 2010. A student who graduated in 2002 has had up to 16 semesters to enroll if

observed in 2010, whereas a student who graduated in the second semester of 2009 has

had –if observed in 2010 -only two semesters to enroll. Figure 2 illustrates the odds of

not enrollment in college by ACCES eligibility. The odds of not enrolling are very high

during the first semesters, as only 5 percent of the eligible and non-eligible individuals

around the cutoff point have enrolled in college one year after high school graduation.

The figure also shows that the risk of not enrolling remains very high even five years

after graduating from high school. Finally, the risk of not enrolling is slightly higher for

the non-eligible students-as just over 20 percent of them have enrolled after 16 semesters.

<<Figure 2>>

Predictors

ELIG is a time-invariant dichotomous variable that indicates whether the student

was eligible to participate in the ACCES (grant and loan) program. If ELIG=1 this means

that the student had a SABER 11 score above the one required in their respective

department of origin and cohort of graduation to qualify for an ACCES grant and loan; if

ELIG=0 the student’s SABER 11 score was below the cutoff for the department.

Assignment to the program was not solely determined by the SABER 11 score, but also

by other factors, such as being displaced. Nonetheless, a very small proportion of the

13

students who apply qualify for these additional points and there is no reason to believe

that the proportion of displaced would be different around the cutoff point. To address the

question of when the student is more at risk of enrolling in college we included time

dummies- SEM- to identify the particular semester to which each row of person-period

refers. For example, SEM1 equals 1 in any row indicating that the student was enrolled

the semester right after graduating from high school, SEM2 equals 1 indicating that the

student was enrolled during the second semester. We included an interaction term

between Eligibility and Semester to explore how time affected the effect of the program

on the Eligible students.

Assignment Predictor

DIST, is a time-invariant continuous predictor that represents the distance

between the students’ SABER 11 score and the cut score for their respective departments

(Figure 1 and Table A2). We included the total distance, the positive distance, the

negative distance, and the square terms of positive and negative distance to explore non-

linearities in the functional form as is the norm in RDD studies (Imbens & Zajonc, 2011;

Reardon & Robinson, 2010).

Finally, we included controls for demographic characteristics available in our

dataset. Some model specifications use fixed effects for departments. As stated in the

model (Equation 1), h(ti) is the population hazard which reflects the “risk” of enrolling in

a postsecondary institution in a specific semester after high school completion,

parameters of β2 to β16 represent the population log-odds of enrolling in the university for

the first time in each semester after high school completion. ELIG, is the dichotomous

treatment indicator (eligible or not to the ACCES program). The two parameters of

interest are β1, which represents the ITT effect of the program, and γi, which determines

whether being eligible has different values according to the number of semesters after

high school completion. We argue that these are precise estimates of the ITT for the

students at the margin of eligibility. We included additional covariates that, according to

the literature, increase the efficiency of the estimate (Trochim, 1984; Murnane & Willett,

2011).We also included fixed-effects to control for other factors related to each specific

geographic region of the country. In addition, as stated by Lesik (2006) there are two

additional threats to the validity of the estimates when using RDD within a discrete time

14

survival model. These are the timing (when) and proportion (amount) of the treatment

received. In terms of the timing, if the students who enrolled in the first semester were

different from the ones who did not enroll the first semester, this could introduce bias in

the treatment effect. We addressed this by including the interaction terms between

treatment variable and semester of enrollment. The proportion of treatment received is

not an issue in our case given that we only test the impact of being eligible on enrollment.

Results

Characteristics of ACCES eligible students at the margin

We start by comparing the characteristics of all the low-income students (SISBEN

1 and 2) who were eligible to receive ACCES with those who actually received the

financial assistance (see Table 1). ACCES is still a relatively small program in the

country. We report that of over 950,000 low-income students who were eligible for a loan

and grant during the period of study, only 3.4 percent actually benefitted from the

program. As reported in Table 1 in this study, we focus on 97,520 low-income students at

the margin (either five percent above or below the cut scores for the program).

Specifically, we report information related to demographic characteristics and enrollment

trajectory for a sub-sample of students at the margin. We first report the differences

between the characteristics of all the low-income students (i.e., SISBEN 1 and 2) in the

study compared to those who received ACCES. The information suggests that there are

indeed differences in the demographic characteristics between these two groups, and that

the differences are statistically significant. We then report the characteristics for the sub-

sample of interest; five percent of the students just above or below the ACCES cut score.

These two groups are very similar in terms of demographic characteristics and that the

actual magnitude of the variables is very similar for all of the characteristics of interest.

For example, the proportion of females, age, and mother’s level of education is almost

identical. This suggests that comparisons between these two groups would probably not

be biased. Finally, the fact that the differences remain statistically significant is mainly

explained by the large sample sizes of our study.

We also report the demographic characteristics for the two groups by department

of the country (see Table A.3). As one would expect, the program is much larger in

Bogota D.C., the capital of the country, and in other major cities or regions of the country

15

(i.e., Antioquia, Atlantico, Bolivar, Cordoba, Cundinamarca, Narino, Norte de Santander,

Santander y Valle), as a result of having larger populations of eligible students. However,

as illustrated in Figure 3, the eligibility criterion is much more stringent in departments

with larger populations of eligible students like Bogota, the capital. The average SABER

11 (maximum 100) for eligible students in Bogota is 43.5 compared to 38.4 in Choco.

The most important information is that the demographic characteristics of the two groups

are very similar within eligible students just above and below the cut score at department

level, which suggests strong internal validity of our findings.

<<Figure 3>>

ACCES has the potential to increase college enrollment of low-income students

We report the results of the different specifications of the hazard models in Table

2.9 The main result is that all the coefficients of the variable of interest, being eligible for

the program, are positive and statistically significant for all the different model

specifications. The standard errors (in brackets) are very small, suggesting that the

coefficients were precisely estimated. This effect is relatively stable to the different

specifications of the model. We also found a positive and statistically significant

coefficient for the interaction term between eligible and second semester (Table 2

Column 7). This means that effect of the program had a greater impact exclusively on

students who had not enrolled in college two semesters after high school completion. The

fact that none of the other coefficients of the interactions terms were statistically

significant suggest that the effect of the program seems to work better for high school

graduates who have not enrolled two semesters after high school graduation than for

students who either have recently graduated or would take more than two semesters to

enroll in college. In general, our findings suggest that the program helps to boost higher

education enrollment of low income individuals as the results show a positive and

significant coefficient for eligible students at the margin.

To estimate the magnitude of the impact of the program on eligible students, we

use the coefficients from Table 2 (column 6) to predict the proportion of eligible

individuals who would have enrolled in college since the existence of the ACCES

program (Figure 4). To determine the enrollment rate of eligible students each semester

9 We also reported the marginal effects of the program on Table A.4.

16

after high school completion (bold line), we estimate the fitted hazard probabilities of not

enrollment (survival rate) with the following formula:

) h (t j )

1

1 e() 1ELIG1 .

) 2SEM 2 ...

) 16SEM16

) 17DIST17

) 18DISTPos18

) 19DISTNeg19

) 20DISTPossq20

) 21DISTNegsq21

) 22DEM 22

) 23FIXEDEF23 )

(2)

The estimated enrollment probability for a particular semester is found by

multiplying the estimated enrollment probability of the previous year by one minus the

estimated risk of not enrolling (survival) that particular semester (Lesik, 2007; Singer &

Willet, 2003).

) S (t j )

) S (t j1)[1

) h (t j )] (3)

We then recalculated the estimated enrollment rates under the assumption that the

ACCES loan never existed. Taking only the sub sample of eligible students we found,

had ACCES not been available the enrollment rate of eligible students would have been

18 percent lower (dashed line) –around two percentage points less- than the one observed

under the current scenario with the program (bold line). This suggests that the intent to

treat (ITT) of the program was positive, meaning that the program has contributed to

increase the number of low-income students enrolling in a postsecondary institution.

<<Table 2>>

<<Figure 4>>

Robustness Checks

We conducted two robustness checks, first, a falsification test using the hazard

logit model and, second, estimates from the Box-Cox proportional hazard models along

with a falsification test.

As for the falsification test using the hazard logit model10 (Table 3), we arbitrarily

moved the cut score by five percent. In our original setting we normalized the ACCES

cut score to zero and we restricted the analysis to the five percent above (eligible) and

below (non eligible) this cut score. In the falsification test we moved the cut score to five

10 Recently a number of studies in economics have used falsification tests to check the robustness of their findings using different rigorous quasi-experimental methodologies. See Sabia & Burkhauser, (2008),

17

percent. So now, we assumed that the five percent below the cut score were not eligible

(even though they were), and the five percent above the cut score were eligible. Thus, we

then made the assumption that those five percent below the false cut score were not

eligible. We expected to find a non-statistically significant coefficient. As reported in

Table 3, this is exactly what we found after we introduced departmental fixed effects in

the models. So we are now in a position to rule out other non-observable factors related

to our findings. In addition, with the exception of the coefficients for the two

demographic factors (i.e., gender and age), none of the other coefficients were

statistically significant. In summary, the falsification test suggests that the effects that we

found are related to being eligible for the program and not to other non-observable factors

unrelated to the program.

We also present estimates using a proportional Box-Cox model11 that allows us to

determine the impact of being eligible for ACCES loan on college enrollment. Unlike

the discrete time hazard models, these models, report the proportional hazard—namely,

the ratio of hazard rate-- for the groups under comparison. The findings, reported in Table

4, indicate that being eligible results in a higher hazard -and therefore a shorter time of

non-enrollment – after controlling for other variables. For instance, being male results in

a lower hazard and hence a longer time of non-enrollment. The Box-Cox estimates

indicate that the proportion of enrollment of the ACCES eligible students would have

been between 14 and 17 percentage points higher compared to students at the margin who

were non-eligible. The magnitude of the coefficient for eligible increases noticeably with

the introduction of additional control variables. The differences of hazard rate between

eligible and non-eligible is quite similar to the one we found by using the discrete time

hazard model. These are ITT impacts for the students at the margin and should not be

compared directly with estimates of about three percent related to receiving an additional

$1,000 in the form of grants (Dynarski, 2002). The falsification tests with the Box-Cox

models are also very robust. Thus, after controlling for the square of distance and

departmental or regional fixed effect, the impact of being eligible compared to being

(falsely) non-eligible is statistically non-significant. Again, as in the falsification test

11 This is a way of determining the optimal transformation of a model in the presence of non-normal or censored data (Stata Corporation, 2011).

18

under discrete time hazard model the only significant coefficients (column 5) are two

demographic factors (age and gender).

Taken together, the findings of our study suggest that the loan program for low

income students was well designed by the ICETEX as it has contributed to increase the

enrollment of low income students in college.

<<Table 4>>

Conclusions and Policy Implications

Colombia has one of the oldest and more developed public financial aid programs

in Latin America. Currently about 20 percent of students benefit from financial aid. To

the best of our knowledge, this is the first study to conduct a rigorous evaluation on the

impact of the ACCES program on college enrollment and the results indicate that

ACCES is a well-designed program that has the potential to serve their target population

of students. However, descriptive evidence suggests that low-income students are more

likely to enroll in low-quality private colleges that have higher drop out rates and are

more costly (Gomez & Celis, 2008). The World Bank recently approved a second loan to

the country conditional on the country trying to diversify its funding resources while

continuing to scaling up the provision of students loans from the lowest socioeconomic

strata (World Bank, 2010). The results of the study are relevant not only to researchers

but to policy makers by providing evidence that ACCES was well designed and, with

minor changes, has the potential to continue to increase enrollment of the lowest-income

individuals in the country.

Acknowledgements This study was supported by a small grant by the Spencer Foundation (2011-00128). We would like to thank Juan Esteban Saavedra and Judy Scott-Clayton for comments on earlier versions of the paper.

19

References Becker, G. (1967). Human capital and the personal distribution of income: An analytical

approach. Ann Arbor, MI: University of Michigan Press.

Bloom, H. S., Michalopoulos, C., & Hill, C. J. (2005). Using experiments to assess

nonexperimental comparison-group methods for measuring program effects. In H. S.

Bloom (Ed.), Learning more from social experiments (pp. 173-235). New York: Russell

Sage Foundation.

Cabrera, A.F., Stampen, J.O., & Hansen, W.L. (1990). Exploring the effects of ability to pay

on persistence in college. Review of Higher Education, 13(3), 303-337.

Calcagno, J.C., & Long, B.T. (2008). The impact of remediation using a regression

discontinuity approach: Addressing endogenous sorting and noncompliance. Working

Paper 14194. Cambridge, MA: National Bureau of Economic Research.

Cerdán-Infantes, P., & Blom, A. (2007). Colombia: Assisting students from low-income

families to attend tertiary education. En Breve, (100) The World Bank. Retrieved on

December 1, 2012 from https://openknowledge.worldbank.org/handle/10986/10303

Chay, K.Y., McEwan, P.J., & Urquiola, M. (2005). The central role of noise in evaluating

interventions that use test scores to rank schools. American Economic Review, 95(4),

1237-58.

Chen, R., & DesJardins, S.L. (2010). Investigating the impact of financial aid on student

dropout risks: Racial and ethnic differences. Journal of Higher Education, 81(2), 179-

208.

Cook, T.D. & Campbell, D.T. (1979). Quasi-Experimentation: Design and analysis for field

settings. Chicago: Rand McNally..

Departamento Nacional de Planeación, (2007). Educación superior-generalidades

[Postsecondary education- general statistics]. Retrieved on July 15, 2007

http://www.dnp.gov.co/paginas_detalle.aspx?idp=593

DesJardins, S.L., Ahlburg, D. A., & McCall, B. P. (2006). An integrated model of application,

admission, enrollment, and financial aid. Journal of Higher Education, 77(3), 381-429.

20

Dynarski, S. (2003). Does aid matter? Measuring the effect of student aid on college

attendance and completion. American Economic Review, 93(1), 278-288.

Dynarski, S. (2002). The behavioral and distributional implications of aid for college.

American Economic Review Papers and Proceedings, 92(2), 279-285.

Gaviria, A., & Toro, J.R. (2012). Cobertura y calidad en la educacion superior en Colombia.

Libro Blanco.Gomez, V.M., & Celis, J.E. (2009). Credito educativo, acciones afirmativas

y equidad social en la educacion superior en Colombia. Revista de Estudios Sociales,

(33), 106-117

Hansen, W.L. (1983). Impact of student financial aid on access. In J. Froomkin, (Ed.), The

crisis in higher education. New York: The Academy of Political Science, 84-94.

Heller, D.E. (1997). Student price response in higher education: An update to Leslie and

Brinkman. Journal of Higher Education, 68(6), 624-659.

Imbens, G., & Zajonc, T. (2011). Regression discontinuity design with multiple forcing

variables. Retreived on November 7, 2012 from

http://www.people.fas.harvard.edu/~tzajonc/boundary_rd_latest.pdf

Imbens, G., & Angrist, J. (1994). Identification and estimation of local average treatment

effects. Econometrica, 62(2), 467-476.

Jackson, G.A. (1988). Did college choice change during the seventies? Economics of

Education Review, 7(1), 15-27.

Jimenez, E., & Tang, J. (1987). Selecting the brightest for postsecondary education in

Colombia: The impact on equity. Economics of Education Review, 6(2), 129-135.

Kane, T. J. (1994). College attendance by Blacks since 1970: The role of college cost, family

background and the returns to education. Journal of Political Economy, 102(5), 878-911.

Lesik, S. A. (2007). Do developmental mathematics programs have a causal impact on student

retention? An application of discrete-time survival and regression-discontinuity analysis.

Research in Higher Education, 48(5), 583-608.

21

Lipsey, M. W., & Garrard, W. M. (2007). Evaluating a school-level statewide conflict

resolution education program for middle schools. Paper presented at the annual meeting

of the Society for Prevention Research, Washington, DC.

Long, B.T. (2004). How have college decisions changed over time? An application of the

conditional logistic choice model. Journal of Econometrics, 121, 271-296.

McPherson, M.S., & Schapiro, M.O. (1991). Does student aid affect college enrollment? New

evidence on a persistent controversy. American Economic Review, 81(1), 309-318.

MEN - CEDE. (2009). Deserción en Educación Superior. [Dropout rates in higher education].

Ministerio de Educación Nacional (MEN) y Centro de Estudios sobre Desarrollo

Económico (CEDE), Universidad de Los Andes. Retrieved on July 29, 2009 from

http://spadies.uniandes.edu.co/spadies2/recursos/Deserción_Educación_Superior.pdf

Murnane, R.J., & Willett, J.B. (2011). Methods matter: Improving causal inference in

education and social science research. Oxford: Oxford University Press.

Reardon, S., & Robinson, J.P. (forthcoming). Regression discontinuity designs with multiple

rating-score variables. Journal of Research on Educational Effectiveness.

Sabia, J.J., & Burkhauser, R.V. (2008). The effects of minimum wage increases in New York

state: Evidence from a natural experiment. Working Paper.

Schochet, P.A. (2005). Statistical power for random assignment evaluations of education

programs. Princeton, NJ: Mathematica Policy Research.

Shen, H., & Ziderman, A. (2008). Student loans repayment and recovery: International

comparisons. The Institute for the Study of Labor, IZA, Discussion Paper Series, DP. No.

3588.

Singell, L.D. (2004). Come and stay a while: does financial aid effect retention conditioned on

enrollment at a large public university? Economics of Education, 23, 459–471.

St. John, E.P. (1990). Price response in enrollment decisions: An analysis of the high school

and beyond sophomore cohort. Research in Higher Education, 31(2), 161-176.

St. John, E.P., & Noell, J. (1989). The effects of student financial aid on ACCES to higher

education: An analysis of progress with special consideration to minority enrollment.

Research in Higher Education, 30(6), 563-581.

Stata Corporation (2011), Stata Survival Analysis and Epidemiological Tables. Reference

Manual, Release 12, Stata Press Publication, College Station, Texas.

22

Stinebrickner, R., & Stinebrickner, T. (2008). The effect of credit constraints on the college

drop-out decision: A direct approach using a new panel study. American Economic

Review, 98(5), 2163-2184.

Van der Klaauw, W. (2002). Estimating the effect of financial aid offers on college enrollment:

A regression discontinuity approach. International Economic Review, 43(4), 1249-1287.

Wanamaker, M.H. (2010). Did industrialization cause the American fertility decline? Evidence

from South Carolina. Working Paper. Retrieved on November 7, 2012 from

http://www.econ.yale.edu/seminars/echist/eh10/wanamaker-032210.pdf

World Bank, (2012). Reviews of national policies for education: Tertiary education in

Colombia. A World Bank country study. Washington, D.C.

World Bank, (2010). Project restructuring of Colombia: Second student loan support project to

Instituto Colombiano de credito educativo y estudios tecnicos en el exterior (ICETEX).

Report No.: 57452v1. Washington, D.C.

World Bank, (2003). Tertiary education in Colombia: Paving the way for reform. A World

Bank country study. Washington, D.C.

Xu, Y. (2008). Does mortgage deregulation increase foreclosures? Evidence from Cleveland.

Working paper. Retrieved on November 7, 2012 from https://editorialexpress.com/cgi-

bin/conference/download.cgi?db_name=NASM2011&paper_id=638

23

Table 1. Descriptive Statistics and Timing of Enrollment into a Postsecondary Institution in Colombia between 2002 and 2010 Students who qualified for SISBEN 1 and 2 and were eligible for ACCES and who had up to 16 semesters to enroll into a Postsecondary Institution

Source: Author's calculations using information from MEN and SPADIES 2002-2010. 1 Eligibles include all the low-income students (defined by qualifying for SISBEN 1 and 2), who took the high school exit exam and had the final score necessary to qualify for an ACCES loan plus grant in their department. 2 Time in between the student took the SABER11 (high school exit exam) and when they enrolled at a postsecondary institution. We excluded individuals who enrolled in technical or technological colleges.

24

Table 2. Estimates for Regression Discontinuity Hazard Logit Model of College Enrollment Students belonging to SISBEN 1 and 2 and 5% around the cutoff score for ACCES eligibility

Source: Author's calculations using information from MEN and SPADIES 2002-2010. Standard errors in brackets *** p<0.01, ** p<0.05, * p<0.1

25

Table 3. Falsification Test for Regression Discontinuity Hazard Logit Model of College Enrollment Students belonging to SISBEN 1 and 2 and 5% around the FALSE cutoff score for ACCES eligibility

Source: Author's calculations using information from MEN and SPADIES 2002-2010. Standard errors in brackets *** p<0.01, ** p<0.05, * p<0.1

26

Table 4. Estimates for Regression Discontinuity Box-Cox Model of College Enrollment Students belonging to SISBEN 1 and 2 and 5% around the cutoff score for ACCES eligibility

Source: Author's calculations using information from MEN and SPADIES 2002-2010. Standard errors in brackets *** p<0.01, ** p<0.05, * p<0.1

27

Figure 1. Eligibility Rate by High School Exit Exam Score

28

Figure 2. Hazard of Enrollment of Eligible Students between 2002 and 2010

29

Figure 3. Eligibility Rate by Poverty Rate and High School Exit Exam Score

30

Figure 4. Simulation of Proportion who Would Have Enrolled-Hazard Models

31

Table A.1. ACCES Selection Criteria

Source: Phone conversation with Alexandra Hernandez, ACCES project director. Notes: a The ICETEX uses the SABER 11 score of students in the different geographical regions, Departments, and creates an adjusted scale. For example, they create pools of students applying to technical institutes in Bogota, they sort the SABER 11 scores in descending order then the create deciles, they remove the lowest decile, and they create an adjusted score from 95 (highest decile) to 15 (lowest category).

32

Table A.2. Maximum Number/Spot required to be Eligible by Department (1 is the top)

33

Table A.3. Descriptive Statistics and Timing of Enrollment into a Postsecondary Institution in Colombia-Geographical Departments Students who qualified for SISBEN 1 and 2 and were eligible for ACCES and who eventually enrolled into a Postsecondary Institution

34

Table A.4. Marginal Effects -Regression Discontinuity Hazard Logit Model of College Enrollment Students belonging to SISBEN 1 and 2 around the cutoff score for ACCES eligibility Source: Author's calculations using information from MEN and SPADIES 2002-2010.

Standard errors in brackets *** p<0.01, ** p<0.05, * p<0.1

35

Table A.5. Estimates for Regression Discontinuity Hazard Model for College Enrollment- Falsification Tests Students belonging to SISBEN 1 and 2 and 5% around the FALSE cutoff score for ACCES eligibility

Source: Author's calculations using information from MEN and SPADIES 2002-2010. Standard errors in brackets *** p<0.01, ** p<0.05, * p<0.1