Building trust before building a system: the making of the Homeless Information Management System

All government managers want to know how well their programs work, but few have the right information to tell them. Learn how one group of state, local, and nonprofit organizations are changing that situation.

• Introduction
• Feasibility first
• A partnership strategy
• Policy challenges
  • Confidentiality concerns
  • Understanding data use
• Data challenges
  • Data quality and fit for use
  • Using meta data and contextual knowledge to develop rules for data integration
  • Too few or too many data standards
  • Who knows enough to know if the data is usable
• Technological challenges
  • The importance of "up front" work before making technology choices
  • Infrastructure impact on technological choices
• Skills needed
  • Recruit the people who have the skills you really need to succeed
• Cost considerations
  • High cost of transforming data into information
  • The costs of on-going support
• Looking to the future

Introduction
Each night in New York State nearly 29,000 homeless people receive emergency shelter and support services. The 6,400 families and 10,000 single adults require assistance in dealing with their immediate incidence of homelessness as well as assistance in dealing with a variety of other problems including domestic violence, alcoholism or substance abuse, poor parenting skills, mental illness, and a lack of education or employment skills. Many lack the skills to maintain their own housing.

New York State and its localities spend millions of dollars and devote substantial effort in providing both housing and services to these homeless single adults and families. The Bureau of Shelter Services (BSS) manages the temporary housing services program in New York State. The program is comprehensive in that it determines eligibility and need for services, provides case management, direct services, and referrals to outside service providers. The cost to federal, state, and local government programs for the homeless in New York State is estimated to be $350 million annually, of which $130 million is spent on service programs.

The Center for Technology in Government (CTG) worked with the New York State Office of Temporary and Disability Assistance (OTDA), Bureau of Shelter Services (BSS) to devise an integrated system that will help government and nonprofit organizations manage homeless services and evaluate their effectiveness. The outcome of this project was the creation of the Homeless Information Management System (HIMS) which is a prototype that draws upon data from multiple existing case management systems and financial systems. The HIMS data repository allows decision makers at the state, local, and provider levels to manage and evaluate temporary housing and service programs for homeless families and single adults.

Feasibility first
Professionals in the homeless services field believe the various service programs they provide to homeless people reduce public assistance costs by helping people achieve independence. But there is little evidence to either support or challenge this belief. Program managers do have quarterly aggregated statistical reports from shelter and service providers regarding the numbers of people being served for payment purposes. However, information about service effectiveness is mostly anecdotal.

State and local program managers need consistent and complete data across service programs and over time to determine the most effective mix of services for a particular client population. This type of data resides in various separate systems or in paper records that are not integrated. As a result, it is unclear whether self-sufficiency, reduced recidivism, reduced dependence on public assistance, and improved overall life skills are being systematically achieved.

BSS staff share growing government-wide interest in outcome-based assessments as well as growing appreciation for how new technologies can support data integration and access. Accordingly, they began to consider the feasibility of a new information resource to help them assess effectiveness across programs, services, and population groups.

The investigation, conducted as part of CTG’s Using Information in Government Program, included the design and development of a prototype system to help BSS decide if it was:

• feasible to develop an integrated database from such a wide variety of data sources
• possible to accurately match individual client information across multiple systems
• reasonable to create a system that would allow for the integration of external data sources
• realistic to think that effective partnerships could be formed to support the necessary collaborations to ensure HIMS included the necessary data

Eighty percent of the homeless population in New York State resides in New York City, and Westchester and Suffolk Counties. BSS has regulatory oversight responsibility for all nonprofit and local government service providers that receive financial support from the State. In this role, BSS writes the regulations that govern the physical, financial, and program requirements for shelters. It certifies shelter programs according to these requirements and conducts periodic inspections of all shelters.

In NYC, BSS shares this regulatory role with the New York City Department of Homeless Services (NYC DHS) for those providers that also receive funding from the City. Although there are a few City-operated programs, the overwhelming majority of service providers are nonprofit organizations. Some are very small operations serving only a few people or families at a time. Others are major programs of large well-established organizations like the Salvation Army and the American Red Cross. Outside New York City, county social services agencies have similar responsibilities to oversee shelter and service programs.

Read full case

A partnership for strategy
About half of the nonprofit homeless service providers in NYC, Westchester, and Suffolk counties are members of a committee of the shelter providers organization called the Technology Committee. The Technology Committee was formed in 1997 to respond to a new information system for case reporting that was being mandated for use in NYC-based shelters by NYC DHS.

The Technology Committee strongly opposed that system for several reasons. It was a canned commercial system that was selected by DHS without consulting with the shelter providers. The system did not assist providers in case management, but added a new system and reporting responsibility to their existing operations. The system would collect not only demographic information about clients, but case notes, the highly personal information that case workers collect for purposes of working with clients on their individual problems and needs. The Committee took its concerns to the leadership of the provider community, which successfully brought pressure on the City to abandon the effort. Because other information technology questions and opportunities continued to emerge, shelter providers decided to continue this useful forum to jointly address information technology issues.

Given the experience with the City’s case reporting system, BSS staff recognized that the success of any new state system would rest heavily on the extent to which providers supported it. BSS has the authority to mandate compliance with any program it sponsors, but understood that to achieve a high quality shared information resource they had to pursue a collaborative approach.

BSS made significant investments in building relationships and trust in the early stages of the project. Many meetings were held to discuss how this initiative would be different from others. The BSS Director made a personal and organizational commitment to the local government agencies and the provider community. He promised that "if they don't see value in the system as a tool to support individual providers as well as the community as a whole, then it won’t be built." Despite these assurances, the Committee members were very guarded in their participation.

As part of their commitment to partnership, BSS established a regular series of meetings in NYC with the Committee. New York City and Westchester and Suffolk county staff participated in these discussions. Facilitated sessions, led by CTG staff, began drawing out and addressing provider concerns. The broad range of challenges highlighted below covered policy, data, technology, skills, and cost considerations.

Read full case

Policy challenges

Confidential treatment of client information
Shelter providers are in the human services business. Their staff interact daily with people who have a variety of personal problems and needs. Many are trained social workers and a strong ethic of client confidentiality pervades the provider community.

One of the first challenges the Bureau faced was concern from the shelter providers that existing policies would not protect their clients’ confidentiality if they shared case management data with BSS. This concern dominated early discussions. During the course of these meetings it became clear that the providers were unaware of the stringent requirements and protections already in use by OTDA for other client-oriented systems such as the Welfare Management System. These are based on the New York State Social Services law which requires the agency protect client confidentiality and limit or prohibit the use of data outside the program for which it is collected.

The Director of BSS compiled these documents and sent them to the committee with a cover letter of assurance from the Commissioner of OTDA. The material cited specific statutes, regulations, guidelines, and procedures that addressed this threshold concern for providers. The combination of formal documentation with a strong legal basis and the assurance of OTDA’s top executive allowed the group to move forward to operationalize these policies.

In this context, several specific concerns emerged. One had to do with unique populations. For the majority of providers, sharing data meant the release and use of client demographics such as name, social security number, age, and address. The Domestic Violence shelter providers had quite different concerns than the rest. Since their clients are in danger of being assaulted or otherwise harmed by people who know them, the most confidential information had not to do with their identity, but with their physical location.

Sharing information that linked a particular client to a particular shelter was therefore of great concern to these providers. The group came to understand that different kinds and levels of data security would be necessary to account for these important differences among programs. In this case, all agreed that the facility information and address had to be masked to protect the location of the client.

Although the concept of HIMS began to gain acceptance among the service providers, it also faced serious problems in obtaining the support of other state agencies.

The Office of Children and Family Services (OCFS), the Department of Health (DOH), the Department of Labor (DOL), the Office of Alcohol and Substance Abuse Services (OASAS), and the Division of Parole also provide services to the homeless population. They have similar difficulties in assessing the impact of the services they provide. These agencies recognized that participating in HIMS might provide them with access to more comprehensive information. However, they were concerned about the same restrictions of sharing data as the local shelter providers. In general, client data cannot be shared without the client's consent. Some agencies receive blanket consent at the point of service application, but others do not.

To further complicate matters, OTDA is one of several agencies created in the 1997 breakup of the former Department of Social Services (DSS). As a result, health care information about homeless clients, once collected and maintained by DSS, had become the province of the Health Department. OTDA and BSS did not have automatic access rights to the Medicaid Management Information System (MMIS) even though the MMIS system itself is linked to and relies on data from OTDA’s Welfare Management System (WMS). BSS has made progress, but not yet succeeded, in securing cooperation from DOH and other agencies by negotiating rules, interpreting existing agreements, and assessing how aggregated data might overcome issues of confidentiality.

Read full case

Understanding data use
While the issues of confidentiality were being addressed, a new policy concern emerged—How would shared data be used? Shelter providers were concerned that BSS would use the data to publicly measure and report their performance as provider organizations. They wanted the data to be used to assess the impact of specific programs independent of the provider. Here again, history played a role. Around the same time, New York City was developing a family shelter incentive program by which providers could receive a bonus of up to 3% of their budgets if they met specified performance goals.

The providers feared that the goals would be unrealistically high and result in negative perceptions of their programs. They carried this concern over to the discussions of HIMS. The BSS staff did not agree to ignore provider-specific information. Instead they pointed out that their existing inspection process already collects the same information, so the risk was no greater with HIMS.

On the other hand, HIMS could offer them important benefits. They would be able to assess their own programs against their peers. And, the ability to compare programs and outcomes across the whole system would identify the best performers which would probably signal best practices that everyone could share. Through these discussions, providers began to see how HIMS could benefit them directly.

That sense of benefit grew when BSS invited them to jointly design a new model of program assessment. Through on-going quarterly meetings, BSS and providers are working to develop a shared framework for program and service evaluation. They are tackling the difficult questions of performance measurement and trying to define, for example, what kind of action, behavior, or outcome constitutes a "success" for a deeply troubled individual compared to a relatively stable family. A simple head count says nothing about these questions. The group began by specifying how HIMS might tell them which services lead to the best outcomes for different categories of clients.

Through meetings, presentations, conference calls, and one-on-one discussions with providers, BSS generated trust that information the providers share with the state will not be used to threaten the well-being of clients or used against specific providers or program managers. In this more trusting atmosphere, the group was able to turn its focus to the practical questions of usability and value of HIMS.

Data challenges
The proposed integrated repository would test the feasibility of obtaining data from disparate sources, and accurately matching the data so it could be aggregated and analyzed to evaluate services. Demographic data was needed from the homeless service providers who maintained client information in their case management systems. Payment information came from the State's legacy Welfare Management System (WMS), and facility information was provided from BSS’s provider certification database. Ideally, medical information would eventually come from the State Health Department’s Medicaid Management Information System (MMIS) and data on substance abuse or other services would come from other state agencies. The prototype design team set out to integrate some of these data sources into a secure Web-enabled system that could be used by all participants.

Four provider organizations from the Technology Committee (Homes for the Homeless, HELP-USA, the Salvation Army, and NYC Human Resource Administration Office of Domestic Violence & Emergency Intervention) volunteered to provide data needed to develop the prototype. The provider data pertaining to family shelters, the data from the BSS facility file, and individual client data from WMS were used to create the Homeless Information Management System prototype.

Read full case

Data quality and fitness for use
As the providers began sharing their data, the extent of data quality issues quickly became a concern. Data quality problems have many causes. The most obvious problem is a data entry error. This is typically addressed through internal data entry procedures and audit checks. BSS and its partners, however, faced much more complex and less tractable problems.

One common source of data errors is the stressful situation of the client at the point of entry to a shelter. The decision to go to a shelter is frequently a last resort for a client. The primary concern for a domestic violence client is to stay hidden from an abuser. Some clients have severe mental health or substance abuse problems that make it impossible for them to provide needed information. In some cases, clients may deliberately provide false information in order to protect their anonymity. More commonly, the stress associated with the situation causes clients to forget or have no record of dates, social security numbers, and past histories. Thus the information provided to the case manager at intake can be fraught with gaps and errors. Case managers may choose not to collect all required information in one session. Several providers said they may take up to two or three weeks to complete all the basic information in a client record. In many cases data for a client remains incomplete in some respects.

In an ideal situation, the providers would have the capacity to match client data against a master system, such as the Welfare Management System or New York City's Human Resource Administration system, to verify or complete missing data. However, that was not feasible in this case. Clients are entered under various names, names are misspelled in multiple ways, social security numbers (when used) are often incorrect, and family composition can vary from one date to the next.

For example, one household record contained a female client with two dependent children when they entered the shelter. The next day they moved to another facility and family composition was recorded as a female head of household and three children. Was this a data entry error or was it a correct reflection of the state of this household? As it turned out, both entries were correct. One child was living with a grandparent and reunited with her family on the second day. This is a fairly common pattern. Children are sometimes placed in foster care or with family members while temporary housing is found. Once placement has occurred, the children are reunited with their families.

Data quality tools have limited capacity to address such unstructured quality issues. Data quality tools focus on auditing, migrating, or cleansing the data based on predesigned business rules (e.g. Name = alpha field, 12 characters in length, value = text string). In HIMS, every record in the prototype was reviewed by the design team and discussed at length with the data provider so key concerns and specific errors could be addressed. Some errors were easily corrected while others needed to be researched with program managers or data technicians.

In the end, all provider data sets were scrutinized, error reports were generated, and data inclusion and exclusion rules were developed. Some data gaps were filled by sending BSS staff into the field to read case records and record the missing elements. Some of the data was "cleaned" with review or filter programs, but this was a minor part of the effort. The human intervention was essential and time consuming—and it required extensive knowledge of the complex program environment. This costly process was feasible only because the prototype data sets were very small. In a fully operational system, much more standardization among the data sources will be required.

Read full case

Using meta data and contextual knowledge to develop rules for data integration
The challenge was not only in obtaining the data but also in finding commonality among the data elements in the Homeless Information Management Systems (HIMS) project. The design team needed to understand how the data was collected, what similarities existed among the data sources, and how the data was going to be aggregated in the new system. This required business rules and standards for the new integrated system as it related to the questions the new system hoped to address. While this seemed easy at the beginning, the true complexity emerged as the team wrestled with such seemingly simple terms as ‘age,’ ‘ethnicity,’ and ‘admit-date.’

The challenge is illustrated well by the process of deciding how a client’s age would be calculated. The decision did not lie with conventional data definitions—in most transactional systems a person’s age would be calculated based on their birth date and system date. However in this system, the age would need to be based on business rules for how a client would be profiled; would age be based on date of entry, on age when referred to a service, or age when a service is rendered or completed? Each decision had to be based on how the data was going to be used in the aggregated form and what questions the system would be used to answer.

This effort involved more than the technical staff. Each question had to be considered from the program and business perspectives as well as the technical perspective. Because the system was being created as a data mart, the transformation (aggregation) of the data was a crucial factor. The extent of aggregation determines the depth of the data mining capabilities in the future system. The lowest level of aggregation for each data field had to be considered. For example, if age were aggregated by ten-year groupings (e.g. birth-10 yrs old, 10-20 yrs old, 20-30 yrs old) we would not be able to a profile clients who are 18 years of age, an important milestone birthday for many public programs.

For each data element, the team had to agree on common definitions and consider how these definitions would affect the inclusion or exclusion of data elements into the integrated system. Each decision made needed to be revisited with each additional data source. These questions helped define the business rules that shaped the system. While they were easily addressed from either a data management or a technology perspective, the more global policy perspective was both more difficult and more important. It not only provided the policy framework for the entire system, but also assured that the system would provide data that would support informed decision making.

Read full case

Too few or too many data standards?
Provider organizations are structured in several ways. Some providers are single-site facilities, others are part of a large corporate nonprofit organization. Not surprisingly, the extent of investment in development and use of data standards varies from one organization to the next. Some have extensive case management systems, others less sophisticated systems, and still others only manual paper records. Each provider has a different data dictionary and naming convention for specific data elements. And, each has individualized business rules that dictate what types of data are collected for each element.

For example, the fieldname ADMITDATE exists in multiple provider databases. In one provider's system, ADMITDATE refers to the day a client entered the shelter, while in another the ADMITDATE was used for both the date a client entered the shelter and the date a room or facility assignment changed. For the second provider, a client might have multiple ADMITDATE entries. Since ADMITDATE is used to calculate a client’s length of stay in a shelter, the difference in these two rules is important. Length of stay information is used to calculate payments and determine recidivism rates. BSS and the providers are continuing to work on an approach, involving both data definitions and algorithms, that will allow for this critical information to be available, as well as reliable and usable.

The HIMS design rested on the ability to establish a unique identifier for every record within the integrated system. Unfortunately, but not surprisingly, each provider assigns a different unique identifier to a client. Originally, the design called for the use of a client’s social security number (SSN) as the identifying code. SSN is recorded in some database systems, but some providers never ask for this information.

Certain providers know that even if their clients have a social security number, it is unlikely that they would recall it during the intake process. Some providers use an identification number assigned by NYC DHS, while others assign a system-specific identification number. The WMS legacy system assigns a Client Identification Number (CIN) at intake but also collects SSNs and Human Resources Administration (HRA) Case Numbers. When domestic violence is an issue, the social security number is recorded but may be changed to protect the client. The domestic violence database maintains no cross-reference system so the social security number cannot be used to match clients in the domestic violence database with any other database.

The design team had the task of developing a common identifier for the prototype or a specific procedure so that data could be cross-referenced with WMS and across provider systems. The design team found that each record contained either the social security number or the HRA number for the head of household. Once this was discovered, the team was able to match either number in the WMS system to obtain a client’s CIN number. The CIN number can also be used in the future to match to other legacy systems so that medical assistance and public assistance information can be obtained. To do this, a matching program had to be written for each system feeding the prototype. While labor intensive, it could be reused by each provider once the initial program was generated.

Read full case

• cleansed the data
• created a consistent structure and format for use across all systems
• created a data dictionary that explained the type of data that was to be brought from each provider’s database for each field
• had access to all of the original data

It was still difficult to completely understand the data and its potential use. In addition to the data consistency issues discussed above, each data code for the integrated system needed to be reviewed in the context of related programmatic issues. In some cases, data was collected based on unique policies or business rules specific to the provider. For example, each system contained information regarding a client’s ethnicity. Usually ethnicity had five categories, while in one instance the provider registered 12 different categories. Conventional wisdom would collapse the 12 into five generic codes and then the five codes would be used in the integrated system. However, the 12 ethnicity categories were extremely important to that provider because they are tied to federal regulations and funding requirements for its programs. Therefore, the system design needed to incorporate a translation table that would feed data to HIMS, while retaining the ability to provide data back to the provider without losing the provider’s expanded categories.

Read full case

The HIMS system is not designed to replace or replicate a daily transaction process or the case management systems used within the provider community. HIMS is to provide a historical view of the impact of service programs on the homeless community.

The design and development of this type of system differs from the traditional On-line Transaction Processing systems (OLTP) in that it is not transaction based. It is historical in nature and relies on a design process referred to as On-line Analytical Processing (OLAP). OLAP software allows users to quickly analyze aggregated information into multi-dimensional views or hierarchies. It can answer such questions as: "What is the average age of a client in facility X for time period Y?" These multi-dimensional views, frequently called "cubes of data" can then be "sliced and diced" to allow variations on the original question: "In facility X, how many 18-21 year old females were "first time" residents during October - December, 1999?"

The importance of "up front" work before making technology choices

The Bureau of Shelter Services staff approached the choice of technology platform from the perspective of their business needs rather than based on specific hardware or software preferences. Through facilitated discussion sessions, the project team outlined what capabilities they envisioned this new resource to have. They did not discuss features such as "data warehouse," "SQL Server," or "processing speed of….." Instead, they discussed a number of business process attributes such as: "ability to analyze public assistance and homeless services," "standard template for correspondence," "uniform definition of services," "matching to external files," "ability to do visual analysis and exception reports," and "remote access with appropriate levels of security."

Those capabilities were grouped in a framework of modest, moderate, and elaborate features and functionality. These categories corresponded roughly to the lowest level of functionality that was worth pursuing, to a more robust set of features with more benefits (and costs), to the most extensive system that they could reasonably expect to justify. BSS chose to pursue the moderate level of system functionality, which would meet essential current needs and allow for eventual expansion to include some of the elaborate level features.

Read full case

Many had never had access to a PC let alone the Internet. Those who did have PC capabilities often had either no access to the Internet or had policies limiting the access to the Internet. Those providers who were expected to purchase in-house case management systems in the near future were also limited in their knowledge and capabilities to make such a purchase. Few had resources on which to call. Overall the BSS team found the majority had limited funds, staff, and knowledge on how to access such a system.

BSS staff adopted a developmental strategy to address this situation. First, they started by working with whatever data is available, usually from the larger nonprofits and the local DSSs, including New York City. Second, they are helping the provider community find and encourage software developers to listen to their needs and develop low-cost, easy-to-use, homeless-oriented systems that over time will improve the technological capacities and data resources of the remaining organizations.

Read full case

Skills needed

Recruit the people who have the skills you really need to succeed
The design team was intentionally made up of program as well as technology experts. Each brought a unique and valuable perspective to the project. As described earlier, it was imperative to have both involved to assist in the definition of the new system as it related to data, policy, and technology decisions. This joint capacity allowed the team to make decisions quickly and detect gaps in the decision-making process. The provider community’s practical perspective gave the team the ability to address operational and policy issues as soon as they were identified. Information management skills such as the identification of data sources, data collection, data security issues, data repository methodologies, and quality control techniques were all necessary in developing HIMS.

At different phases of the project, different skills were needed, and different team members were added to the mix. As technologists or business analysts were needed they were brought in. As their roles were completed, their activity diminished. The one consistent facet of the project team that never changed was the involvement of the BSS staff. They provided the managerial as well as the program focus of the team. Each staff member, acting as liaison to the provider community, could address the goals and the challenges of the project. This provided the continuous, consistent communication that was so important in building and maintaining trust with the providers.

Read full case

Cost considerations
Traditionally, estimating the cost of a new system takes into consideration the initial design and start-up costs. The development costs (such as hardware, software, and consulting) and the production system costs (such as hardware, software, and dedicated staff) are readily quantifiable. However, few consider the cost of the staff time required in the early stages of problem definition and relationship building. These are often hidden costs that do not go into the investment calculations, yet they are essential, substantial, and continuous.

Getting early relationships into a more trusting mode and constantly reinforcing these relationships consumed large portions of the Bureau’s time and attention. Regular large group meetings, weekly status meetings, reaching out to potential data providers in other agencies, and building working relationships with other divisions of OTDA were all costly, but essential project activities.

Read full case>

High cost of transforming data into information
As each data set was added to HIMS, the human intervention required to review each data question was multiplied. The project team, comprising 10 people, spent seven months in what is called the discovery phase of system development. The discovery phase is characterized by many discussions of the specific business process the system is designed to support and the business rules it will follow.

The actual prototype development, the technology component of the project, took two months time for two developers, a majority of which was spent on data transportation and transformation rather than creation of the actual application.

Both BSS and provider staff, along with expert consultants reviewed data sets to address:

• data quality issues
• data transportation issues (moving the data sets to a staging area in preparation for transformation)
• data transformation issues (changing data sets based on new business rules governing the new system)
• data exportation issues (moving the new aggregated data sets from the staging area to the new system)

This process absorbed an enormous amount of project staff and consultant time. As discussed above, the contextual knowledge required during this phase was imperative to ensure correct business decisions were made. Substantial time was spent in drafting new business rules and making data inclusion decisions—time seldom considered in the cost estimation models.

Read full case<

The cost of on-going support
Few cost estimation models take into consideration what it will cost to support a new initiative past the initial implementation. The BSS team outlined the initial internal costs required to support the new system, as well as what they believed their user community would need to participate in the on-going requirements of the system. Part of the estimation reflected the realization that during the prototype testing few providers had sufficient equipment and expertise to participate. BSS looked for ways to help the provider community obtain the minimally required hardware and software to participate in the program. In their initial budget request, BSS factored these initial start-up costs in as part of the administrative costs incurred by each facility.

BSS staff and local providers will also need assistance with the day-to-day use of the new system. Where will the providers go for assistance? The BSS staff is very small. They must consider how they will provide support not only in the development, but also the on-going maintenance of the new system. What on-going role can the Technology Committee play? These factors also need to be considered in estimating the cost of a full system.

Read full case

Looking to the future
The HIMS prototype was evaluated by a team of state staff and shelter providers in September 1999. Based on their positive assessment, BSS adjusted its design and prepared a budget request to go forward with a full system development project. The staff have the full support of the agency’s leaders and have identified several sources of funding to help support the project. BSS has also joined the Technology Committee in the search for effective case management systems that will help shelter providers manage their programs and collect data in a way that makes it easier for them to share that data with HIMS in the future.

Committee meetings now focus on system demonstrations and discussions that are moving toward these goals. Providers are actively supporting the BSS budget request and looking for resources of their own that will make them better able to participate in HIMS when it comes online. Most important, as a group, these professionals are armed with essential knowledge about the challenge that lies before them. Their expectations are high, but tempered by the difficulty of the environment they work in and the complications it presents.