Barrier Free Living
Barrier Free Living (BFL) is an award-winning non-profit organization that has worked with survivors of domestic violence with disabilities since 1981, and has received CJII support for its work with Deaf/deaf/Hard of Hearing survivors of domestic violence. Historically, a large proportion of BFL Freedom House residents do not fully participate in on-site Occupational Therapy workshops (OT).
The NYC Behavioral Design Team (BDT) partnered with BFL to increase the number of residents who attend OT workshops by redesigning three important OT marketing and outreach materials: 1) a weekly schedule, 2) a single event flyer, and 3) a daily reminder email. The BDT incorporated a number of behavioral design tactics into each piece to increase the salience of the benefits of participating, highlight social norms, and make it easier to remember session offerings. Furthermore, the BDT developed a comprehensive Behaviorally Informed Communications Best Practices guide to help the BFL team integrate these best practices into all of their communications efforts.
Hunter College, ISLG, and the Manhattan DA’s Office (DANY)
Mayor’s Fund to Advance NYC
- Finalized deliverable (evaluation report) and disseminated results in convening with NYC and partner stakeholders in November 2021.
Mayor’s Office of Resiliency (MOR)
- Language from the mailer was adapted and now lives on FloodHelpNY, a website for engaging NYC homeowners about how they can protect their home and finances from flooding.
- A randomized controlled trial showed a 4.6 percentage point increase in survey completion, which means households were 15.5x more likely to complete the survey if they received our “last chance letter.”
- The “last chance letter” was scaled to every household eligible for the affordability study.
- Personalized action steps were delivered to every household eligible for the affordability study.
New York City 311
- Pre/Post evaluation showed that changing menu choices and reframing language around feedback decreased misfiled service requests from 59% to 9%.
New York City Administration for Children’s Services (ACS)
- The redesigned behavior management system (“STRIVE+”) is currently being implemented in every secure youth detention centers in NYC and staff are being trained in its principles.
New York City Department of Consumer and Worker Protection (DCWP)
The City has important tax information to share with residents, but it often lacks the contact information to do so. To increase sign-ups for tax preparation messaging, the BDT redesigned the Tax Prep Portal website to reduce ambiguity around services and eligibility. The redesigned website used icons to display information in a digestible way, social norms messaging to enhance relevance, and default options to reduce choice conflict.
- While a randomized controlled trial was attempted to measure the impact of the redesigned website, the study was underpowered.
- The redesigned website was scaled citywide for future tax years.
In response to the temporary expansion of the Child Tax Credit, the BDT worked with the NYC Department of Consumer and Work Protections (DCWP) Office of Financial Empowerment (DCWP OFE) to help over 250,000 low-income NYC families access these enhanced benefits. Targeting families who receive SNAP or cash assistance and did not submit taxes in 2019, the BDT designed a behaviorally informed postcard to encourage recipients to file their taxes, claim unclaimed Child Tax credits, and use the free tax preparation services.
- The postcards were translated into eight languages and mailed to families in April 2022.
New York City Department of Education (DOE)
- A randomized controlled trial showed that the redesigned email increased testing requests by 5.0% among all districts and 9.0% among low-income school districts.
- Engagement among students was high: 81% of participating students responded at least once.
- The revised materials were rolled out in Fall 2021.
New York City Department of Finance (DOF)
- A randomized controlled trial showed that individuals who received our behaviorally informed letter were 6.7 times more likely to take steps to resolve their summons debt.
- A randomized controlled trial showed that expensive tickets belonging to drivers that received the behaviorally designed envelope 90-days after receiving their ticket were 9.7 percentage points less likely to enter into judgment.
- The DOF scaled the behaviorally designed envelope to all drivers whose unpaid tickets prompt the 90-day notice in August 2018.
- A randomized controlled trial showed the redesigned email reduced boot eligibility by 10.4%.
- The DOF scaled the redesigned email to all drivers approaching the booting threshold.
- 60-day notices were scaled citywide.
- A randomized controlled trial found no statistically significant effect, likely due to low power. However, 30-additional businesses registered after receiving the behavioral email compared to the business-as-usual email.
- Mailed inserts were scaled citywide.
New York City Department of Health and Mental Hygiene (DOHMH)
Asthma medication administration forms (MAFs) allow school staff to administer asthma medication to students in respiratory distress, reducing emergency room visits and lost classroom time. MAFs must be completed by a physician and signed by a parent/guardian, but in 2014-15 only 43% of approximately 70,000 diagnosed students had MAFs on file. To encourage submission, the BDT designed and tested a new MAF that was mailed to physician offices and supplemented with a call. The new design reduced the number of steps on the form, presented all information for physicians on the front and for the parent on the back, and added arrows to make salient the most important information (i.e. parent’s signature). Because physicians access documents online, only a few physicians’ offices used the new form. After conducting post-mortem interviews and surveys with physicians, the BDT learned that physicians widely approve of the new form.
- The behaviorally designed MAF was scaled to all physicians citywide, with improved distribution in 2017-18.
Continuing its work on increasing the submission of MAFs, the BDT conducted qualitative interviews with community school case managers, school physicians, and school nurses. To simplify what is asked of parents, the BDT created a simple and personalized slip (sent home with the form) with clear instructions on how and when to complete and submit the form. This slip implements a self-affirmation note that reinforces parents’ identities as good parents. Additionally, the BDT created a decision tree that simplified responsibilities for school nurses to ensure that students can receive emergency medication while in school.
- The self-affirmation notes and decision trees were scaled to all NYC school nurses at the start of the 2017-18 school year.
School health forms are essential to ensuring their children have access to the care they need to stay healthy while in school. Without these forms, school health providers may lack necessary information about a student’s health status or be unable to administer important medical treatments. Unfortunately, many families do not complete and submit these forms on time – if at all. The BDT partnered with the Office of School Health within the Department of Health and Mental Hygiene (DOHMH) to increase the timely submission of health forms. Together, we designed three communications to increase the salience of forms and decrease the hassles of submission: 1) an infographic for school staff with tips to help them reduce hassles and cognitive burden on families completing health forms, 2) a letter from the Office of School Health that answers common questions and makes the benefits of submission salient, and 3) a template letter for schools to use to introduce their school nurse to build a sense of trust and provide families a checklist to support form submission.
- The designs were rolled out to the NYC public school system in the spring of 2022.
The Health Justice Network (HJN), a project of the New York City Department of Health (DOHMH), helps connect people returning from jail and prison to primary care and other resources. The NYC BDT partnered with HJN to audit HJN's outreach materials and design a new flyer for the program, leveraging behavioral design best practices and insights from conversations with Community Health Workers about the context in which HJN participants decide whether to participate in the program and take up primary health care services. The redesigned materials aim to make it easier for potential participants to learn about the benefits of the program and take the first step towards participation.
New York City Department of Housing Preservation & Development (HPD)
- Recommendations informed the design of the new NYC Housing Connect online portal, which launched in Summer 2020 and includes a behaviorally-informed income calculation guide and housing suggestions tailored for applicants.
New York City Department of Small Business Services (SBS)
- The email and cover page were implemented citywide for all eligible businesses.
New York City Economic Development Corporation (NYCEDC)
- Recommendations have been incorporated into materials created for the first cohort of students.
- Following the first cohort, we will work closely with NYCEDC and SBS to design and test further ways to optimize the program design.
New York City Fire Department (FDNY)
- A randomized controlled trial showed that fee waivers increased registration amongst female candidates by 83% and registration amongst black candidates by 84%.
- Registration amongst male and white candidates also increased, but only by 22% and 38%, respectively.
- A randomized controlled trial showed that candidates who received both a text message and an email were 3.6% more likely to appear for the exam than those who received the other forms of outreach.
- The survey closed in November 2018, and final results were presented in April 2019.
- Survey responses demonstrated that Black and female candidates express experiencing threats related to their race and gender, respectively, regardless of whether or not they were hired.
- FDNY, in response, is in the process of leveraging opportunities to reduce perceived threat throughout the hiring process. For example, FDNY’s future communication materials and outreach/programming for candidates are designed to reduce perceived threat.
- A randomized controlled trial showed no statistically significant difference in claimed points for those who received social norms messaging compared to loss aversion messaging.
- A small sample of underrepresented candidates who received a more intensive campaign was 7.4 percentage points more likely to claim residency points, which is statistically significant. This suggests that a more intensive behavioral campaign targeting encourages candidates who are less likely to claim their points.
- The reflection exercise and planning card were rolled out to candidates in June 2019, and are now administered as part of the training curriculum for all candidates who attend an information session prior to the start of the CPAT Training Program.
New York City Human Resources Administration (HRA)
SNAP churn (losing benefits, then returning within 90 days) hurts recipients and burdens HRA. To improve SNAP recertification rates, the BDT designed a reminder notice spurring clients to take the first required recertification step: submitting a recertification form. The notice leveraged loss aversion by reframing recertification as a potential loss of benefits and used graphics and numbering to make action steps clear and seem more manageable.
- A randomized controlled trial showed the reminder reduced failure to submit recertification forms by 5.5%
- The reminder notice was scaled to all clients who do not submit their forms by the reminder mailing date.
New York City offers several housing assistance programs to households at risk of becoming homeless. To prompt the uptake of valuable housing assistance, the BDT redesigned a notice sent to at-risk households urging them to call the City’s Neighborhood Homelessness Prevention Outreach (NHPO) program for potential assistance. The redesigned notice used a testimonial to reframe services as an opportunity benefitting others and included a plan-making prompt to help households carve out time in their day to call.
- A randomized controlled trial showed no statistically significant differences in call-in rates or rent assistance between the original and the redesigned notices.
Many SNAP clients fail to complete the required recertification interview and, as a result, lose their benefits. To improve SNAP recertification rates, the BDT redesigned a notice sent to SNAP recertification clients who had submitted their recertification forms but had yet to complete their interviews. The redesigned notice clearly outlined the time commitment and materials needed for the interview and included a planning prompt for calling at a later date and time.
- A randomized controlled trial showed no statistically significant difference in interview completion rates between the original and the redesigned notices.
SNAP churn results from failure to recertify annually but is also driven by a failure to complete the six-month periodic report. The BDT designed two behaviorally informed emails to prompt online periodic report submission. Both emails leveraged loss aversion, outlined action steps and time expectations, and made deadlines salient. One email also used enhanced active choice.
- The enhanced active choice emails increased engagement rates by 30.1%.
New York City is the first city in the country to enact a “universal-access-to-counsel” (UATC) law that provides residents facing eviction in Housing Court with free legal assistance. To raise awareness and increase utilization of these valuable services, the BDT worked with the Human Resources Administration’s Office of Civil Justice to redesign outreach materials, such as flyers, that are sent to New Yorkers facing eviction court proceedings advising them of the resources available and encouraging them to access these services. The behaviorally designed flyer used social norms messaging to highlight other people’s use of the program, outlined the wide range of situations in which people could benefit from having a lawyer or accessing legal guidance, and provided an actionable next step for receiving legal services. These designs more clearly delineated for readers when and how lawyers are useful, more effectively empowering tenants to act if and when they feel at risk.
- A randomized controlled trial showed no statistically significant difference in uptake of legal assistance between the original and the redesigned flyers.
Many Human Resources Administration (HRA) clients visit Job or SNAP Centers in person to complete tasks they could more conveniently complete using ACCESS HRA (AHRA), an app and website that allows New Yorkers to manage their benefits digitally. To increase client uptake of AHRA, the BDT is working with HRA to design a set of interventions to be implemented in Centers. A digital tools area will provide clients with an appealing, low-stakes opportunity for learning about and trying digital tools. This intervention aims to help clients better understand the benefits of AHRA, overcome ambiguity aversion, and establish a habit of using AHRA to manage their benefits. A staff training will increase staff’s understanding of AHRA, and also promote positive staff-client interactions to maximize staff’s ability to engage clients in learning about the digital tools while in the Center.
- The digital tools area and staff training were deferred due to the COVID-19 pandemic.
Due to the COVID-19 pandemic, all Human Resources Administration (HRA) staff supporting the SNAP program were teleworking for the first time. The BDT worked with HRA to increase worker motivation and efficiency and engage workers in an iterative design process to create new worker-facing electronic dashboards. These dashboards were designed to distribute daily tasks across the SNAP processing team and provide staff with more consistent, transparent feedback on their performance. They incorporated behaviorally informed design elements such as motivational framing to emphasize the positive impact of staff’s work on client well-being and improve staff efficiency and accuracy.
- The new system, known as ANGIE, is being piloted in 2021-22.
FairFares NYC offers low-income NYC residents a 50% discount for the subway and bus system, as well as a 50% discount for New Yorkers with disabilities or health conditions who use Access-A-Ride paratransit services. A significant number of Fair Fares enrollees and applicants who attempt to link their Fair Fares NYC account with Access-A-Ride are found ineligible, most frequently because they enter an invalid Access-A-Ride ID. The BDT worked with HRA to identify the behavioral drivers that lead participants to enter an invalid Access-A-Ride ID for the discount. Based on our findings, we designed changes to the online application process that aimed to reduce the number of unmatched applicants, and ultimately ensure prospective applicants are matched to the programs for which they desire to apply their discount.
- A pre-post analysis found that the application changes were associated with a 109% increase in the proportion of link attempts that were successful.
New York City's Department of Social Services (DSS) designed Current NYC, a new Rental Assistance and Landlord Management System, to make it easier for staff to manage clients’ rental assistance applications and rent payments to landlords. The BDT worked with DSS to design a series of three emails to introduce the new system to staff, aiming to build buy-in and help staff know what to expect during the pilot and full rollout of the system.
- Current NYC has been rolled out to all relevant staff and rollout to providers is ongoing through spring 2023. The emails are being sent to staff and providers as they onboard.
HRA's new "One Number" Interactive Voice Response System will merge 35+ legacy customer service phone lines into a single phone line. The BDT partnered with HRA to conduct user testing with representatives of community-based organizations who work directly with HRA clients, and provide input on the design of the new system. The BDT’s activities ensured that the menu design incorporates behavioral design best practices and is responsive to client needs and advocate feedback.
New York City Office of Labor Relations (OLR) + WorkWell NYC
- Behavioral emails doubled click-through and registration rates. The enhanced active choice version of the email increased vaccine uptake at worksite locations by 5%.
- The most successful arm was scaled in 2017-18 and a 10% increase in vaccine uptake was observed year over year.
- Email designs were scaled citywide after the piloted email blast enrolled about three times as many people as the first email.
- A pseudo-random assignment was designed, but no comparative data were released by the insurer.
- Design recommendations will be incorporated into citywide newsletters, communications, and upcoming programming.
New York City Taxi and Limousine Commission (TLC)
- A randomized controlled trial showed drivers who received the behaviorally designed mailing were 15% more likely to enroll in the program, but the underpowered study did not find this difference statistically significant.
- The behaviorally designed mailing was scaled to all eligible taxicabs.
New York City Young Men's Initiative (YMI)
The City University of New York (CUNY) Fatherhood Academy (CFA) is a 16-week program funded by the Young Men’s Initiative (YMI) that supports fathers ages 18-30 living in New York City with attaining their High School Equivalency (HSE) or enrolling in college. The BDT partnered with YMI and CFA to design and pilot test behavioral designs to increase enrollment by reducing hassles, leveraging positive identities, and countering negative expectations. Designs included an enrollment journey map, text message reminder series, Instagram post, and behavioral updates to CFA’s website and intake form.
- Findings from the pilot showed positive qualitative impacts on enrollment and designs were rolled out across all three campuses in January 2022.
The Door is a multi-service youth center (and CJII Youth Opportunity Hub) located in lower Manhattan and the south Bronx serving young people from all over New York City. Historically, a significant proportion of people who sign up as members at The Door do not enroll in any of the organization’s specific programs and services (though they may informally take part in various offerings). The NYC Behavioral Design Team (BDT) partnered with The Door to increase the number of young people who participate in programming. Based on its qualitative research, the BDT designed a series of resources for young people to use during their initial sign-up to narrow down service options and guide them through enrollment.
- All three designs were adopted by The Door in May of 2022. We are currently awaiting quantitative data to inform our evaluation of the designs.