Hunter College, ISLG, and the Manhattan DA’s Office (DANY)

The Navigators Program leverages a grassroots network of Community Navigators to identify community members' needs and connect them to vital services. The program defines these connections to City agencies and CBOs as “referrals,” and Navigators work with participants through coaching and regular check-ins to meet their housing, benefits, education, legal, and other referral needs. This project centered on understanding the number of “referrals” or successful connections participants made to agencies with Navigator assistance. The BDT found that while Navigators are assisting New Yorkers with connecting to a multitude of imperative services, Navigators were unclear as to which services and for whom such services should be logged as “referrals,” therefore affecting accurate program performance. The BDT identified behavioral solutions to increase correct data capture to ensure accurate measurement and evaluation of all the work Navigators perform.

NYC Human Resources Administration (HRA)

SNAP churn (losing benefits, then returning within 90 days) hurts recipients and burdens HRA. To improve SNAP recertification rates, the BDT designed a reminder notice spurring clients to take the first required recertification step: submitting a recertification form. The notice leveraged loss aversion by reframing recertification as a potential loss of benefits and used graphics and numbering to make action steps clear and seem more manageable.
  • A randomized controlled trial showed the reminder reduced failure to submit recertification forms by 5.5%
  • The reminder notice was scaled to all clients who do not submit their forms by the reminder mailing date.
Learn More
New York City offers several housing assistance programs to households at risk of becoming homeless. To prompt the uptake of valuable housing assistance, the BDT redesigned a notice sent to at-risk households urging them to call the City’s Neighborhood Homelessness Prevention Outreach (NHPO) program for potential assistance. The redesigned notice used a testimonial to reframe services as an opportunity benefitting others and included a plan-making prompt to help households carve out time in their day to call.
  • A randomized controlled trial showed no statistically significant differences in call-in rates or rent assistance between the original and the redesigned notices.
Many SNAP clients fail to complete the required recertification interview and, as a result, lose their benefits. To improve SNAP recertification rates, the BDT redesigned a notice sent to SNAP recertification clients who had submitted their recertification forms but had yet to complete their interviews. The redesigned notice clearly outlined the time commitment and materials needed for the interview and included a planning prompt for calling at a later date and time.
  • A randomized controlled trial showed no statistically significant difference in interview completion rates between the original and the redesigned notices.
SNAP churn results from failure to recertify annually but is also driven by a failure to complete the six-month periodic report. The BDT designed two behaviorally informed emails to prompt online periodic report submission. Both emails leveraged loss aversion, outlined action steps and time expectations, and made deadlines salient. One email also used enhanced active choice.
  • The enhanced active choice emails increased engagement rates by 30.1%.
Learn More
New York City is the first city in the country to enact a “universal-access-to-counsel” (UATC) law that provides residents facing eviction in Housing Court with free legal assistance. To raise awareness and increase utilization of these valuable services, the BDT worked with the Human Resources Administration’s Office of Civil Justice to redesign outreach materials, such as flyers, that are sent to New Yorkers facing eviction court proceedings advising them of the resources available and encouraging them to access these services. The behaviorally designed flyer used social norms messaging to highlight other people’s use of the program, outlined the wide range of situations in which people could benefit from having a lawyer or accessing legal guidance, and provided an actionable next step for receiving legal services. These designs more clearly delineated for readers when and how lawyers are useful, more effectively empowering tenants to act if and when they feel at risk.
  • A randomized controlled trial showed no statistically significant difference in uptake of legal assistance between the original and the redesigned flyers.
Many Human Resources Administration (HRA) clients visit Job or SNAP Centers in person to complete tasks they could more conveniently complete using ACCESS HRA (AHRA), an app and website that allows New Yorkers to manage their benefits digitally. To increase client uptake of AHRA, the BDT is working with HRA to design a set of interventions to be implemented in Centers. A digital tools area will provide clients with an appealing, low-stakes opportunity for learning about and trying digital tools. This intervention aims to help clients better understand the benefits of AHRA, overcome ambiguity aversion, and establish a habit of using AHRA to manage their benefits. A staff training will increase staff’s understanding of AHRA, and also promote positive staff-client interactions to maximize staff’s ability to engage clients in learning about the digital tools while in the Center.
  • The digital tools area and staff training were deferred due to the COVID-19 pandemic.
Due to the COVID-19 pandemic, all Human Resources Administration (HRA) staff supporting the SNAP program were teleworking for the first time. The BDT worked with HRA to increase worker motivation and efficiency and engage workers in an iterative design process to create new worker-facing electronic dashboards. These dashboards were designed to distribute daily tasks across the SNAP processing team and provide staff with more consistent, transparent feedback on their performance. They incorporated behaviorally informed design elements such as motivational framing to emphasize the positive impact of staff’s work on client well-being and improve staff efficiency and accuracy.
  • The new system, known as ANGIE, is being piloted in 2021-22.

NYC Department of Education (DOE)

Students in low-income districts are less likely than their peers in higher-income districts to take the Gifted & Talented (G&T) admissions test. The BDT redesigned emails and postcards to encourage preschool parents to sign their students up for the test. The redesigned email used a congratulatory tone to prime the positive identity of a proud parent and used icons and succinct language to highlight key information. The postcard reframed the G&T program as one in which all types of students participate and included a sample test question to reduce ambiguity around the testing level.
  • A randomized controlled trial showed that the redesigned email increased testing requests by 5.0% among all districts and 9.0% among low-income school districts.
Learn More
Many high school seniors are accepted to college but neglect to enroll in the months following graduation. To reduce this “Summer Melt,” the BDT designed and implemented a series of two-way text messages during the summer of 2016. The text messages pulled various psychological levers, including plan-making to keep prospective students on track for enrollment.
  • Engagement among students was high: 81% of participating students responded at least once.  
Every Student, Every Day (ESED) is a Community Schools attendance campaign that helps schools use low-cost and scalable strategies to meet the needs of children and remove the barriers to regular attendance. By adopting ESED, schools are poised to reduce chronic absenteeism (both in-person and remotely) and help students succeed in school and in life. The BDT conducted a communications and process audit of selected training and program implementation materials. To complete this audit, we drew upon psychological insights about how people process, remember, and act on information and applied best practices to ensure clear, persuasive, and effective communications and processes.
  • The revised materials were rolled out in Fall 2021.
NYC Department of Consumer and Worker Protection

NYC Department of Consumer and Worker Protection (DCWP)

The City has important tax information to share with residents, but it often lacks the contact information to do so. To increase sign-ups for tax preparation messaging, the BDT redesigned the Tax Prep Portal website to reduce ambiguity around services and eligibility. The redesigned website used icons to display information in a digestible way, social norms messaging to enhance relevance, and default options to reduce choice conflict.
  • While a randomized controlled trial was attempted to measure the impact of the redesigned website, the study was underpowered.
  • The redesigned website was scaled citywide for future tax years.
NYC FDNY

New York City Fire Department (FDNY)

Historically, women and minority group members have become firefighters at lower rates than white men. In the 2017 firefighter recruiting season, the NYPD set explicit targets for increased firefighter diversity and engaged the BDT to design and test ways to increase the number of female and minority candidates entering the process. The BDT targeted the first step in the recruitment process: filing to take the civil service exam. Waiving a small fee associated with filing overcomes a bureaucratic hassle, which may disproportionately discourage minorities and women from filing for the exam.
  • A randomized controlled trial showed that fee waivers increased registration amongst female candidates by 83% and registration amongst black candidates by 84%.
  • Registration amongst male and white candidates also increased, but only by 22% and 38%, respectively.
Learn More
Many individuals who file to take the DCAS Firefighter Exam often fail to appear for their scheduled exam. FDNY engaged with the NYC BDT to test the effectiveness of a reminder campaign, using a randomly-assigned email, text, email and text, and automated voice response (AVR) phone campaign. Based on previous research on plan-making, the BDT redesigned these communications to include a planning prompt to encourage candidates to make a plan for how they would arrive on time for their scheduled exam.
  • A randomized controlled trial showed that candidates who received both a text message and an email were 3.6% more likely to appear for the exam than those who received the other forms of outreach.
Women and underrepresented group members in the past have attrited from the New York Fire Department’s hiring process at higher rates than white males. BDT designed a survey to understand the barriers that female and underrepresented candidates face using psychometrically validated questions that test hypotheses about how relevant psychologies, such as belonging uncertainty and stereotype threat, may influence persistence through FDNY’s hiring process.
  • The survey closed in November 2018, and final results were presented in April 2019.
  • Survey responses demonstrated that Black and female candidates express experiencing threats related to their race and gender, respectively, regardless of whether or not they were hired.
  • FDNY, in response, is in the process of leveraging opportunities to reduce perceived threat throughout the hiring process. For example, FDNY’s future communication materials and outreach/programming for candidates are designed to reduce perceived threat.
Many firefighter candidates with long-term residency in NYC who take the written firefighter exam fail to claim the additional points offered to long-term NYC residents – this difference of a few extra points often bumps candidates into eligibility. To encourage candidates to claim residency points by email, the BDT designed an email and text message campaign, using social norms and loss aversion to grab candidates’ attention, as well as reduced the number of steps required to claim points by drafting a pre-populated email for candidates to easily complete and send to the Department of Citywide Administrative Services (DCAS). Half of the candidates received the social norms messaging while the other half received the loss aversion messaging.
  • A randomized controlled trial showed no statistically significant difference in claimed points for those who received social norms messaging compared to loss aversion messaging.
  • A small sample of underrepresented candidates who received a more intensive campaign was 7.4 percentage points more likely to claim residency points, which is statistically significant. This suggests that a more intensive behavioral campaign targeting encourages candidates who are less likely to claim their points.
Learn More
Among aspiring firefighters applying to join the New York City Fire Department (FDNY), women are nearly twice as likely to withdraw their candidacy just prior to the physical exam as their male peers. Women report being primed to think about negative stereotypes about their gender in this physically demanding setting, which may cause self-doubt and poor performance despite being fully qualified for the job. To address barriers that prevent women from showing up to and performing on their physical exam, the BDT developed an exercise for candidates to reflect on their motivations for becoming a firefighter and a planning card to help candidates form personal training plans and track their weekly progress.
  • The reflection exercise and planning card were rolled out to candidates in June 2019, and are now administered as part of the training curriculum for all candidates who attend an information session prior to the start of the CPAT Training Program.
Learn More

Department of Finance (DOF)

The Criminal Justice Reform Act (CJRA) shifted several low-level offenses (also known as ECB violations) from the criminal courts to the civil courts; however, only 2-3% of such cases get paid after they are docketed by the Department of Finance (DOF). Individuals with unresolved fines may face financial penalties and possible collection activities, which can be damaging to their credit. To prevent financial consequences of non-payment, the BDT designed a letter to individuals with unpaid cases docketed by DOF, encouraging them to pay. The letter primes the recipient’s identity as a member of the community and offers a 2-week deadline extension to encourage reciprocity. It also clearly states the reason DOF is contacting the recipient to highlight the letter’s relevance and includes icons and clear action steps to prompt action.
  • A randomized controlled trial showed that individuals who received our behaviorally informed letter were 6.7 times more likely to take steps to resolve their summons debt.
About a fifth of parking tickets issued to drivers in NYC enter into judgment, among which over three-quarters could have been addressed sooner. When unaddressed tickets enter into judgment, they incur additional fees for the driver. To reduce the number of unpaid tickets entering judgment, the BDT redesigned the envelope containing penalty notices sent to drivers with unpaid tickets. The envelope tested the use of a non-blaming tone to reduce the tendency to ostrich from unpleasant news, as well as attempted to reduce present bias by highlighting the potential losses as a consequence of delaying payment of the ticket.
  • A randomized controlled trial showed that expensive tickets belonging to drivers that received the behaviorally designed envelope 90-days after receiving their ticket were 9.7 percentage points less likely to enter into judgment.
  • The DOF scaled the behaviorally designed envelope to all drivers whose unpaid tickets prompt the 90-day notice in August 2018.
Learn More
When NYC drivers approach $350 in unpaid parking fines and interest, their vehicles become at risk for booting. To release a boot, drivers often pay over $700—an unanticipated financial cost that can be particularly disruptive for those living in low-to-moderate income households. To motivate drivers to take action and lower the risk of booting, the BDT redesigned a warning email delivered to drivers approaching the $350 threshold. The redesign makes salient the consequences of inaction to reduce procrastination, cuts hassles to make payment easier, and reframes the email in a personalized, non-blaming tone to reduce aversion associated with the ticket.
  • A randomized controlled trial showed the redesigned email reduced boot eligibility by 10.4%.
  • The DOF scaled the redesigned email to all drivers approaching the booting threshold.
Learn More
Tens of thousands of property owners in New York City are at risk of having a lien put on their properties, due to non-payment of property taxes. To reduce the number of property owners who are at risk of losing their property, the BDT redesigned the 60-day notice that the DOF sends to these clients. The redesigned notice clarified what the tax lien sale is, where to find more information, and what options property owners have for payment. In addition, the notice used loss aversion and a deadline to inspire action, as well as included a checklist of simplified steps for requesting a payment plan.
  • 60-day notices were scaled citywide.
In 2016, only 6% of licensed businesses on record were registered for the City’s online tax portal. To prompt registration among business owners and tax preparers, the BDT redesigned two emails encouraging registration. The redesigned emails presented social norms messaging about businesses registered on the portal, provided a concrete estimate for the time required for the task, and used icons to highlight the key benefits of online tax filing.
  • A randomized controlled trial found no statistically significant effect, likely due to low power. However, 30-additional businesses registered after receiving the behavioral email compared to the business-as-usual email.
Homeowners in NYC are often surprised by or concerned with their Notice of Property Value (NOPV), which notifies them of their property’s market and assessed values for the coming tax year. To reduce confusion and improve engagement, the BDT designed a behaviorally informed mailing insert to accompany the NOPV for every homeowner in NYC in February 2017 (tax year 2017-2018). The notice clarified what the NOPV is, how to decipher it, and what action (if any) a homeowner should take.
  • Mailed inserts were scaled citywide.

311

The 311 Mobile App team was receiving a large number of misfiled service requests in an early version of the application and wanted improved, more frequent feedback on the app’s functionality from users. The BDT conducted qualitative research with a small sample of users and provided behaviorally informed recommendations to redesign the smartphone app’s menu and reframe its feedback function.
  • Pre/Post evaluation showed that changing menu choices and reframing language around feedback decreased misfiled service requests from 59% to 9%.

NYC Department of Housing Preservation & Development (HPD)

The NYC affordable housing lottery online application system (“Housing Connect”), which currently hosts close to 2 million users, faced a “goodness of fit” problem, or a mismatch between which lotteries applicants apply to and whether their household is ultimately eligible for and interested in those housing developments. To support clients in accurately reporting annualized income and selecting good fit lotteries, the BDT provided a set of behavioral design recommendations that considered key barriers clients face when visiting Housing Connect. Recommendations sought to improve the choice architecture of the portal, making strong matches more salient and reducing the hassles, ambiguity aversion, and present bias that clients encounter with the current design.
  • Recommendations informed the design of the new NYC Housing Connect online portal, which launched in Summer 2020 and includes a behaviorally-informed income calculation guide and housing suggestions tailored for applicants.
Learn More

NYC Administration for Children’s Services (ACS)

The complexity and contradictions within ASPIRE, the behavior management system used at ACS’s secure youth detention centers in NYC, had caused an uneven application of rules by staff and a lack of buy-in amongst youth residing in centers. To encourage youth’s investment in the system, reduce unsafe behaviors, and increase prosocial behavior, the BDT collaborated with ACS to redesign the points and levels structure and revise incentives. Design recommendations were based upon the input of youth and staff in secure centers and addressed behavioral barriers of limited attention, scarcity, and present bias.
  • The redesigned behavior management system (“STRIVE+”) is currently being implemented in every secure youth detention centers in NYC and staff are being trained in its principles.
Learn More

Mayor’s Office of Resiliency (MOR)

As sea levels rise, an increasing number of New York City homeowners are vulnerable to risks associated with flooding—just a few inches of flood water can cause tens of thousands of dollars of damage. To promote homeowners’ flood resilience, The Mayor's Office of Resiliency worked with the BDT to help increase flood insurance enrollment among those living in flood-prone areas. The BDT developed a behaviorally-informed mailer that addresses some of the barriers to flood insurance enrollment. The mailer makes flood risk more salient to homeowners, clarifies complex insurance terminology, provides helpful rules of thumb to homeowners when purchasing a policy, and reduces perceived hassles in the process of purchasing flood insurance.
  • Language from the mailer was adapted and now lives on FloodHelpNY, a website for engaging NYC homeowners about how they can protect their home and finances from flooding.
 
Only 11% of NYC households in flood-prone areas had responded to an online survey, which provided valuable information to inform policies and programs that address expanding flood maps due to climate change. In order to boost engagement with targeted homeowners, the BDT worked with ORR to design and test–via a randomized controlled trial—a behaviorally-informed "last chance letter" that simplified language, depicted the concrete benefits of participation, and imposed a soft deadline.
  • A randomized controlled trial showed a 4.6 percentage point increase in survey completion, which means households were 15.5x more likely to complete the survey if they received our “last chance letter.”
  • The “last chance letter” was scaled to every household eligible for the affordability study.
In a final effort to gather information on the flood resiliency and insurance affordability among flood-risk households, the BDT supported the design of door hangers to accompany door-to-door canvassing efforts to increase uptake of a survey. The design incorporated elements from an earlier randomized controlled trial with ORR: using icons to emphasize the benefits of participation, highlighting a soft deadline, and simplifying the steps to take action. In addition, individuals were given their unique PIN for survey completion on the doorhanger.
  • Personalized action steps were delivered to every household eligible for the affordability study.

NYC Department of Small Business Services (SBS)

While there are benefits to being certified as a Minority & Women-Owned Enterprise (M/WBE), many business owners never pursue or do not renew their certification. To increase the number of M/WBEs certified with the City, the BDT led a behavioral design workshop with SBS’s M/WBE team and co-designed a behaviorally-informed cover page for the application, as well as a behaviorally-informed, mobile-optimized email reminding already-certified businesses that they are required to recertify to maintain their status. The email implemented social norms messaging, simplified action steps, and made salient the benefits of certification.
  • The email and cover page were implemented citywide for all eligible businesses.

NYC Economic Development Corporation (NYCEDC)

Quality job training programs can prepare New Yorkers for good-paying and in-demand careers in technology. However, many of these programs are inaccessible to low-income students due to high upfront costs and limited or inequitable financing. To overcome these barriers and promote economic mobility of New Yorkers, NYCEDC has partnered with NYC Opportunity, the Department of Small Business Services, and the Department of Consumer Worker Protection to design the Talent Financing Fund (TFF), which increases access to high-quality, outcomes-oriented training by offering a new way to pay for vocational training with fair financing terms and critical wraparound services. ideas42 collaborated with NYCEDC to behaviorally audit TFF’s recruitment, application, and enrollment procedures and materials and proactively identify behavioral barriers that may prevent interested New Yorkers from persisting through the process. Recommendations focused on affirming participants' sense of belonging, emphasizing the program’s safeguards and potential benefits, and building an understanding of the financing mechanism.
  • Recommendations have been incorporated into materials created for the first cohort of students.
  • Following the first cohort, we will work closely with NYCEDC and SBS to design and test further ways to optimize the program design.

NYC Department of Health and Mental Hygiene (DOHMH)

Asthma medication administration forms (MAFs) allow school staff to administer asthma medication to students in respiratory distress, reducing emergency room visits and lost classroom time. MAFs must be completed by a physician and signed by a parent/guardian, but in 2014-15 only 43% of approximately 70,000 diagnosed students had MAFs on file. To encourage submission, the BDT designed and tested a new MAF that was mailed to physician offices and supplemented with a call. The new design reduced the number of steps on the form, presented all information for physicians on the front and for the parent on the back, and added arrows to make salient the most important information (i.e. parent’s signature). Because physicians access documents online, only a few physicians’ offices used the new form. After conducting post-mortem interviews and surveys with physicians, the BDT learned that physicians widely approve of the new form.
  • The behaviorally designed MAF was scaled to all physicians citywide, with improved distribution in 2017-18.
Continuing its work on increasing the submission of MAFs, the BDT conducted qualitative interviews with community school case managers, school physicians, and school nurses. To simplify what is asked of parents, the BDT created a simple and personalized slip (sent home with the form) with clear instructions on how and when to complete and submit the form. This slip implements a self-affirmation note that reinforces parents’ identities as good parents. Additionally, the BDT created a decision tree that simplified responsibilities for school nurses to ensure that students can receive emergency medication while in school.
  • The self-affirmation notes and decision trees were scaled to all NYC school nurses at the start of the 2017-18 school year.

Taxi and Limousine Commission (TLC)

An incentive program that pays taxicab drivers to drive Wheelchair Accessible Vehicles (WAVs), aimed at increasing the number of accessible cabs on the road experienced low enrollment and engagement. This prompted the TLC to look into their strategies for engaging drivers, and together the TLC and the BDT redesigned materials mailed to a subset of drivers, including enrollment instructions, a sample application, and a letter, all intended to reduce the hassles and ambiguity of applying to the program.
  • A randomized controlled trial showed drivers who received the behaviorally designed mailing were 15% more likely to enroll in the program, but the underpowered study did not find this difference statistically significant.
  • The behaviorally designed mailing was scaled to all eligible taxicabs.

Office of Labor Relations (OLR) + WorkWell NYC

Within the New York City workforce, over 400,000 employees and their family members were unvaccinated from the flu in the 2015-16 flu season, risking their own health and potentially spreading the flu to others. To increase the uptake of flu vaccines and maintain a healthy workforce during the 2016-17 flu season, the BDT partnered with WorkWell NYC to design a set of three behaviorally-informed emails to all of the NYC employees encouraging them to protect themselves and others from the flu by getting a vaccine. One version of the redesigned email used the strategy of “enhanced active choice,” whereby employees were prompted to make a choice that had a clear right answer.
  • Behavioral emails doubled click-through and registration rates. The enhanced active choice version of the email increased vaccine uptake at worksite locations by 5%.
  • The most successful arm was scaled in 2017-18 and a 10% increase in vaccine uptake was observed year over year.
Learn More
Telemedicine is a low-cost option for insured employees to engage in quality healthcare services, and in 2017 only 180 of 588,000 employees and their dependents in the City’s GHI/EBCBS CBP plan had enrolled. To encourage City employees to enroll, the BDT supported the redesign of the second of three emails that the OLR sent to City employees in January and February of 2017.
  • Email designs were scaled citywide after the piloted email blast enrolled about three times as many people as the first email.
 
In 2017 only 0.03% of employees and their dependents in the City’s GHI/EBCBS CBP plan had enrolled in telemedicine, and City health insurance began offering ZocDoc as an option for finding a primary care physician. The BDT designed a behaviorally-informed to-do list encouraging City employees to register for ZocDoc and telemedicine. The to-do list was included in the mailing that sent City employees their new health insurance cards in the summer of 2016. The checklist was sent to employees with last names beginning with the letters A-K and a control version (with plain formatting) was sent to those with last names beginning with the letters L-Z.
  • A pseudo-random assignment was designed, but no comparative data were released by the insurer.
 
WorkWell NYC offers workplace wellness programming, tools, and resources to 380,000 City employees. However, male City employees' participation in programming is disproportionately low. The BDT partnered with WorkWell NYC to understand the behavioral barriers that prevent male NYC employees from signing up for and participating in WorkWell NYC programs. Based on our findings, we shared behaviorally informed design recommendations to make communications more salient and tailored to men, to introduce a referral program and healthy competition to encourage sign-up, and to adapt programming to better fit into men’s personal and professional lives.
  • Design recommendations will be incorporated into citywide newsletters, communications, and upcoming programming.
 

Mayor’s Fund to Advance NYC

The Mayor’s Fund to Advance New York City launched a pilot program to encourage NYC tax filers to apply for a "refund anticipation loan" where filers receive a portion of their tax refund immediately. While such advances typically come with predatory practices, the City partnered with three financial institutions that offered fair terms in this pilot. ideas42 created a communication to encourage uptake of the services that launched at the end of Jan. 2021. ideas42 worked with NYCO and other partners to conduct a process evaluation of the pilot, producing a report assessing the rollout, laying out behavioral barriers that may have prevented uptake, and presenting design options for future iterations of the program.
  • Finalized deliverable (evaluation report) and disseminated results in convening with NYC and partner stakeholders in November 2021.

Cross-Agency Projects

When COVID-19 struck, NYC government agencies had to swiftly adapt programs and policies to meet evolving, urgent client needs and keep clients and staff safe. The BDT provided quick-turnaround support to assist agencies with their COVID-19 response efforts and help agencies communicate these changes to clients and staff in a clear, actionable, and effective way. The BDT supported 11 agencies across 37 different rapid response engagements between March 2020 and June 2021.