Clinical implications and validity of nursing assessments: a longitudinal measure of patient condition from analysis of the Electronic Medical Record (2025)

Abstract

Objectives This study investigates risk of mortality associated with nurses’ assessments of patients by physiological system. We hypothesise that nursing assessments of in-patients performed at entry correlate with in-hospital mortality, and those performed just before discharge correlate with postdischarge mortality.

Design Cohort study of in-hospital and postdischarge mortality of patients over two 1-year periods.

Setting An 805-bed community hospital in Sarasota, Florida, USA.

Subjects 42 302 inpatients admitted for any reason, excluding obstetrics, paediatric and psychiatric patients.

Outcome measures All-cause mortalities and mortality OR.

Results Patients whose entry nursing assessments, other than pain, did not meet minimum standards had significantly higher in-hospital mortality than patients meeting minimums; and final nursing assessments before discharge had large OR for postdischarge mortality. In-hospital mortality OR were found to be: food, 7.0; neurological, 9.4; musculoskeletal, 6.9; safety, 5.6; psychosocial, 6.7; respiratory, 8.1; skin, 5.2; genitourinary, 3.0; gastrointestinal, 2.3; peripheral-vascular, 3.9; cardiac, 2.8; and pain, 1.1. CI at 95% are within ±20% of these values, with p<0.001 (except for pain). Similar results applied to postdischarge mortality. All results were comparable across the two 1-year periods, with 0.85 intraclass correlation coefficient.

Conclusions Nursing assessments are strongly correlated with in-hospital and postdischarge mortality. No multivariate analysis has yet been performed, and will be the subject of a future study, thus there may be confounding factors. Nonetheless, we conclude that these assessments are clinically meaningful and valid. Nursing assessment data, which are currently unused, may allow physicians to improve patient care. The mortality OR and the dynamic nature of nursing assessments suggest that nursing assessments are sensitive indicators of a patient's condition. While these conclusions must remain qualified, pending future multivariate analyses, nursing assessment data ought to be incorporated in risk-related health research, and changes in record-keeping software are needed to make this information more accessible.

Article summary

Background

Nursing budgets constitute a major part of a hospital's operating costs, accounting for some 25% of the total operating budget and 44% of direct care costs.1 ,2 Recent studies have demonstrated that higher staffing ratios of registered nurses are associated with fewer hospital-related deaths, failures to rescue, cardiac arrests, hospital-acquired pneumonia and other adverse events, as well as having positive effects on patients’ safety in intensive care units and in surgical patients.3–6 Increased registered nurse hours spent on direct patient care were associated with decreased risk of hospital-related death and shorter lengths of stay.3–6 In the course of providing direct patient care, nurses assess each patient by physiological system and record those assessments in the electronic medical record (EMR). Once recorded however, nursing notes are not often read by attendants or residents.7

There have been several studies of the relationship between registered nurse staffing and in-hospital mortality.8 ,9 However, no previous studies have demonstrated a direct connection between nursing assessments and patient risk of mortality. Furthermore, while the nursing literature is replete with studies of nursing diagnostic terminology and its standardisation,10–19 there is a paucity of quantitative studies of the validity and clinical implications of nurses’ head-to-toe clinical assessments. In this article we investigate clinical associations of 12 simplified nursing assessments, one for each physiological systems, with both in-hospital and postdischarge mortality data. Our hypotheses are that nursing assessments performed at entry of in-patients are predictors of in-hospital mortality, and nursing assessments performed just before in-patient discharges are predictors of postdischarge mortality.

Methods

This research was initiated in an attempt to understand issues of continuity of care in hospitals. What piqued our interest in nursing assessments is the fact that other than laboratory tests and vital signs, they are the only clinical variables in the EMR that are reflective of patient condition that are not static, and their values are updated regularly. The results of this research are incorporated in the ‘Rothman Index’, which is a new measure of patient condition. This is the first foundational paper in a series of studies related to the scientific basis of the Rothman Index. Approval for the work was granted by the Sarasota Memorial Hospital Institutional Review Board.

Nursing assessment data for the periods 1/2004–12/2004 and 7/2005–6/2006 were extracted from the Electronic Hospital Record at Sarasota Memorial Hospital, an 805-bed community hospital. Our cohort for this study were all patients admitted for any reason during these periods, excluding obstetrics, paediatric and psychiatric patients, which determined the study size of 42 302 inpatient visits. For the discharge study, we had complete data for 39 964 inpatient visits in which the patient was discharged alive. Demographic data and diagnostic data have not been collected for this population; however, our subject community hospital serves a population skewed older than the US average.

In general, nurses’ assessments are entered into the Electronic Medical Record in one of two ways: either the nurse answers a series of detailed questions to document each assessment, such as for respiratory, ‘What are the breath sounds?’, ‘What colour are the nail beds’, etc; or, the ‘charting by exception’ method is used, where the nurse records a simple overall answer of ‘met’ or ‘not met’ for each physiological system, such as ‘Does the patient's respiratory function meet a minimum standard?’ (if the answer is ‘not met’, only then are detailed follow-on questions considered). In this study the binary charting by exception method was used, and the assessments were characterised as either having ‘met’ the standard or having ‘not met’ the standard in each of the following 12 areas: food, neurological, safety, skin, genitourinary, musculoskeletal, respiratory, cardiac, peripheral vascular, gastrointestinal, psychosocial and pain. Definitions of the relevant standards are shown in the appendix. Nursing assessments were generally performed at least once per shift. For each area of nursing assessment, the all-cause in-hospital death rates and mortality OR's, associated with patients’ entry nursing assessments, were computed and the all-cause death rates and mortality OR's, associated with patients’ last assessments prior to discharge, were computed for patients living at the time of discharge with deaths within the time periods 2 days, 30 days and 1 year from discharge. Mortality was established by comparison with the Social Security Administration Death File. The reproducibility of outcome measures between the 2004 data and the 2005–2006 data was assessed using an intraclass correlation coefficient. The data analysis was carried out utilising SAS V.9.2 (SAS Institute, Cary, North Carolina, USA) and Systat V.13 (Systat Corp., Chicago, Illinois, USA).

Results

The population studied had a mean of 4.7 days and median 3.1 days length of stay, with an SD of 5.1 days. An example of the data and associated results for 30-days postdischarge mortality is given in table 1, illustrating our calculations. Results for in-hospital mortality OR's associated with failing an entry nursing assessment are given in table 2 for 42 302 patients of whom 1086 died in the hospital. OR's for deaths within postdischarge periods of 2 days, 30 days and 1 year for 39 964 patients for whom we had data and who were living at the time of discharge are given in table 3. There were less than 0.3% missing data for any result. Although generally, about 90% of patients passed the assessments, in all categories and for both in-hospital and postdischarge deaths, not meeting standards for an assessment resulted in significantly higher death rates than meeting standards and very large mortality OR's, with the single exception of the pain assessment. Except for the pain assessments, all results are statistically significant (p<0.001) and none of the 95% CI's overlap one. The implications of pain assessments are examined in the discussion section below.

To evaluate the agreement between 2004 and 2005–2006 values, first we calculated all the OR's for the 2004 and 2005–2006 subsets of data, and then an intraclass correlation coefficient was calculated comparing all OR's for 2004 to their counterparts from 2005 to 2006 across all categories and all time points (ie, in-hospital and 2, 30, and 365 days postdischarge) and found to be 0.85; values greater than 0.75 indicate excellent reproducibility.20 Thus, there is excellent reproducibility between the results for 2004 and those for 2005–2006 across all our measures of interest.

Discussion

There is a growing body of evidence that suggests that nurses have a separate and identifiable effect on patient hospital outcomes, irrespective of other medical care (cf Kane et al21 ,22 and references cited therein). An important subject area in the literature is whether nursing data collected during the hospital stay can be used to explain commonly studied variables such as costs/charges, lengths of stay and mortality. However, until now, nursing assessments have not been shown to have clinical predictive validity. Our study combines over 40 000 cases, encompassing all admissions for any reason except for paediatrics, psychiatric and rehab, and we have shown that patients failing to meet minimum standards for any nursing assessment had significantly higher in-hospital and postdischarge death rates than patients who did meet these standards, regardless of medical history and diagnosis. This is evidence of the clinical validity of nursing assessments. In fact, the medical literature is replete with studies of critically ill patients for whom variables with mortality OR in the range 2–4 are considered sensitive and important predictors of mortality.23–26 The OR's we report demonstrate that these measures of a patient's functionality are significant and sensitive indicators of a patient's condition. Thus, nursing assessments may aid in physician care and possibly reduce hospital patient mortality, and future risk-related health research ought to consider incorporation of nursing assessments. Further study is clearly warranted.

We have studied deaths without regard to diagnosis, age, gender or severity of illness that occur in-hospital as well as within short (2 and 30 days) and long (1-year) periods of time for two reasons: (1) in order to learn whether there might be similar or different short and long-term relationships of assessments to mortality and (2) as a way to test validity and reliability. We have found that failing a nurse's assessment has both short-term and long-term correlations with mortality. The fact that our results are consistent over short and long periods, over all assessments, and from a medically diverse patient population, demonstrates that nursing assessments have valid and significant clinical implications irrespective of medical histories and diagnoses. While we have not compared survivor demographics with those who died, it is unlikely that such large OR's would be explained by underlying demographic factors like age or race or gender.

One might ask whether the high OR's associated with these assessments may simply represent high-risk diagnoses. We have not built a multiple linear regression model to determine the added predictive power of nursing assessments to other indicators of mortality. However, this is an area of current investigation. Preliminary results, which we shall publish in subsequent studies, show that among patients who were admitted with a diagnosis of congestive heart failure, those who failed any of the food, neurological, musculoskeletal or skin nursing assessments at admission had a mortality OR of 6–9, versus those who passed. Further, failing multiple nursing assessments had a greater implication in terms of in-hospital mortality than failing just one. Generally, patients who fail nursing assessments may well be those who also have serious or even terminal diagnoses. Nonetheless, here we have demonstrated that nursing assessments at both entry and discharge capture the seriousness of the patient's condition irrespective of diagnosis. Given the size of the study, it is clear that these variables are meaningful indicators of both in-hospital and postdischarge mortality.

The highest in-hospital mortality OR's, 7 or greater, are associated with those patients not passing the food, neurological, psychosocial or musculoskeletal assessments. It is important to comprehend the simple clarity of these findings. For example, if a newly admitted patient is not able to move independently in his/her bed, or behaves inappropriately, or cannot chew and swallow food—that patient is almost seven times more likely to die in the hospital than patients who have none of these problems. If a patient is incoherent or is not oriented, that the patient is almost 10 times more likely to die in the hospital than those who are coherent and alert, no matter what the diagnosis may be. In contradistinction, pain is not a significant indicator of mortality. This may be because pain, unlike the other problems indicated by nursing assessments, can often be controlled independent of the patient's general condition, so one does not expect it to have similar clinical implications. Further studies may clarify this issue.

Since we have shown that initial and final nursing assessments contain clinical information, it is reasonable to infer that all nursing assessments (other than pain) gathered throughout the patient's stay in the hospital contain significant clinical information. Nursing assessments are generally recorded every 12 h, and sometimes more frequently, throughout the patient's stay. Therefore, these data represent a changing indicator of patient condition. Thus, the importance of this work is in alerting physicians to a source of longitudinal clinical information about the patient's condition that no static measurement, such as demographics or principal diagnosis, can provide and which is not currently being routinely utilised.

Although our results show excellent reproducibility across two 1-year time periods, we are aware of questions about the reliability of nursing data, specifically intra-rater and inter-rater reliability. However, reproducible and consistent results such as shown here are only possible with reproducible and consistent nursing assessments. One has to conclude that by and large, the nurses get it right, and that they provide an important and valuable tool for assessing patient condition, irrespective of medical diagnosis and history. Thus, it is suggested that hospital physicians make special effort to ascertain whether patients have passed or failed their nursing assessments, a practice not widely followed currently by attendants or residents.7

We have established that simplified nursing assessments, gathered throughout a patient's stay and which are now essentially unused by physicians, have clinical validity. If physicians were to utilise these data, they then would be adding an important and largely ‘new’ source of clinical information to their evaluations. For example, when a physician sees a patient in the hospital, he or she often consults with the bedside nurse. However, this consultation is not always possible or practical. In many cases, consulting with the nurse from the previous shift, or the previous day, or even the day before that, to understand changes in the patient's condition, is not feasible. And even if it were, getting a verbal report on a patient's condition about the previous several days would likely be incomplete. These circumstances make use of nursing assessments, recorded in the EMR, the only effective way to gain access to this clinically relevant information. And though immediate observations are the most important in determining care, the prior observations provide an important and meaningful context, allowing the physician to assess the changes in patient condition. It must be noted however, that current EMR technology does not facilitate quick and easy physician access to nurses’ observations and assessments. Either changes in current record-keeping software or adjuncts to it would make access to this information more accessible.

Conclusions

Entry nursing assessments of in-hospital patients are strongly associated with in-hospital mortality, and predischarge nursing assessments of in-hospital patients are associated with postdischarge mortality. No multivariate analysis has yet been performed, and will be the subject of a future study, and thus there may be confounding factors. However, it is difficult to hypothesise any alternative factors that might confound our results, given the high OR's and the variety, multiple time intervals, and extent of data considered. We infer that these nursing assessments have valid and significant clinical meaning irrespective of medical histories and diagnoses. It is reasonable to infer further that nursing assessments taken throughout a patient's stay are also clinically meaningful. These assessments, which are part of what is termed as the ‘head-to-toe’ patient assessment, and which are a standard part of nursing school curricula, are collected and recorded at all hospitals, and simplified summaries of assessments, as we have analysed, can be constructed. Since nursing assessments are recorded at least every 12 h throughout the patient's stay, they represent a changing indicator of patient condition. Thus, they make available real-time longitudinal sensitivity that no static measurement, such as demographics or principal diagnosis, can provide.

The large OR's suggest that nursing assessments are sensitive indicators of clinical problems during the course of a patient's hospital stay. This compact clinical data source in the EMR is a natural longitudinal source of information, providing physicians access to the insights of nurses as recorded throughout the patient's entire stay. Such dynamic information should allow physicians to improve patient care. While these conclusions must remain tentative, pending detailed multivariate analyses, we believe nursing assessment data ought to be incorporated along with standard diagnoses in future risk-related health research. Current EMR technology does not allow quick and easy access to nurses’ observations and assessments, so changes in current record-keeping software or adjuncts to it will be necessary to make this information more accessible.

Acknowledgments

The authors thank Sarasota Memorial Hospital for making available the data upon which this investigation is based, and for the cooperation of the staff, especially Steven Goodman, Donna Jackson, Patti Reynolds and Denis Baker. Alex Kiss is thanked for checking calculations, a critical reading of the manuscript, and general statistical advice. Peter Herbert, Chief Medical Officer, Yale New Haven Hospital, is thanked for discussions regarding doctors’ current utilisation of nursing assessments. We also thank Susan Horn and Professor Robert Kane for their insights in reviewing the study, which resulted in significant improvements.

Appendix

References

1. ↵
   1. Welton JM,
   2. Fischer MH,
   3. DeGrace S,
   4. et al

   . Hospital nursing costs, billing, and reimbursement. Nurs Econ 2006;24:239–45, 262, 227.

   OpenUrlPubMedWeb of Science

2. ↵
   1. McCue M,
   2. Mark BA,
   3. Harless DW

   . Nurse staffing, quality, and financial performance. J Health Care Finance 2003;29:54–76.

   OpenUrlPubMed

3. ↵
   1. Aiken LH,
   2. Clarke SP,
   3. Sloane DM,
   4. et al

   . Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA 2002;288:1987–93.

   OpenUrlCrossRefPubMedWeb of Science

4. ↵
   1. Aiken LH,
   2. Clarke SP,
   3. Cheung RB,
   4. et al

   . Educational levels of hospital nurses and surgical patient mortality. JAMA 2003;24:1617–23.

   OpenUrl

5. ↵
   1. Needleman J,
   2. Buerhaus P,
   3. Mattke S,
   4. et al

   . Nurse-staffing levels and the quality of care in hospitals. N Engl J Med 2002;346:1715–22.

   OpenUrlCrossRefPubMedWeb of Science

6. ↵
   1. Mark BA,
   2. Harless DW,
   3. McCue M,
   4. et al

   . A longitudinal examination of hospital registered nurse staffing and quality of care. Health Serv Re 2004;39:279–300.

   OpenUrl

7. ↵
   1. Hripcsak G,
   2. Vawdrey DK,
   3. Fred MR,
   4. et al

   . Use of electronic clinical documentation: time spent and team interactions. J Am Med Inform Assoc 2011;18:112–17.

   OpenUrlCrossRefPubMedWeb of Science

8. ↵
   1. Sales A,
   2. Sharp N,
   3. Li YF,
   4. et al

   . The association between nursing factors and patient mortality in the Veterans Health Administration: the view from the nursing unit level. Med Care 2008;46:938–45.

   OpenUrlCrossRefPubMedWeb of Science

9. ↵
   1. Sasichay-Akkadechanunt T,
   2. Scalzi CC,
   3. Jawad AF

   . The relationship between nurse staffing and patient outcomes. J Nurs Adm 2003;33:478–85.

   OpenUrlCrossRefPubMedWeb of Science

10. ↵
    1. Barthold M

    . Standardizing electronic nursing documentation. Nurs Manage 2009;40:15–17.

    OpenUrlPubMed

11. ↵
    1. Bernhart-Just A,
    2. Lassen B,
    3. Schwendimann R

    . Representing the nursing process with nursing terminologies in electronic medical record systems: a Swiss approach. Comput Inform Nurs 2010;28:345–52.

    OpenUrlPubMed

12. ↵
    1. Cheevakasemsook A,
    2. Chapman Y,
    3. Francis K,
    4. et al

    . The study of nursing documentation complexities. Int J Nurs Pract 2006;12:366–74.

    OpenUrlCrossRefPubMed

13. ↵
    1. Jefferies D,
    2. Johnson M,
    3. Nicholls D

    . Nursing documentation: how meaning is obscured by fragmentary language. Nurs Outlook 2011;59:e6–12.

    OpenUrlCrossRefPubMed

14. ↵
    1. Jefferies D,
    2. Johnson M,
    3. Griffiths R

    . A meta-study of the essentials of quality nursing documentation. Int J Nurs Pract 2010;16:112–24.

    OpenUrlCrossRefPubMed

15. ↵
    1. Kol Y,
    2. Zimmerman P,
    3. Sadeh Z

    . Common nursing terminology for clinical information systems. Stud Health Technol Inform 2005;116:629–34.

    OpenUrlPubMed

16. ↵
    1. Laitinen H,
    2. Kaunonen M,
    3. Astedt-Kurki P

    . Patient-focused nursing documentation expressed by nurses. J Clin Nurs 2010;19:489–97.

    OpenUrlPubMed

17. ↵
    1. Langowski C

    . The times they are a changing: effects of online nursing documentation systems. Qual Manag Health Care 2005;14:121–5.

    OpenUrlPubMed

18. ↵
    1. Lee TT

    . Nursing diagnoses: factors affecting their use in charting standardized care plans. J Clin Nurs 2005;14:640–7.

    OpenUrlCrossRefPubMedWeb of Science

19. ↵
    1. Lees L

    . Improving the quality of nursing documentation on an acute medicine unit. Nurs Times 2010;106:22–6.

    OpenUrlPubMed

20. ↵
    1. Rosner B

    . Fundamentals of biostatistics. 5th edn. Boston, MA: Brooks/Cole, 2000:627.

21. ↵
    1. Kane RL,
    2. Shamliyan TA,
    3. Mueller C,
    4. et al

    . Nurse staffing and quality of patient care. Evid Rep Technol Assess 2007;151:1–115.

    OpenUrl

22. ↵
    1. Kane RL,
    2. Shamliyan TA,
    3. Mueller C,
    4. et al

    . The association of registered nurse staffing levels and patient outcomes: systematic review and meta-analysis. Med Care 2007;45:1195–204.

    OpenUrlCrossRefPubMedWeb of Science

23. ↵
    1. Egi M,
    2. Morita K

    . Fever in non-neurological critically ill patients: a systematic review of observational studies. J Crit Care 2012 (in press).

24. ↵
    1. Baptista R,
    2. Jorge E,
    3. Sousa E,
    4. et al

    . B-type natriuretic peptide predicts long-term prognosis in a cohort of critically ill patients. Heart Int 2011;6:e18.

    OpenUrlPubMed

25. ↵
    1. Ribera A,
    2. Ferreira-González I,
    3. Marsal JR,
    4. et al

    . Prognostic value of an abnormal ankle-brachial index in patients receiving drug-eluting stents. Am J Cardiol 2011;108:1225–31.

    OpenUrlPubMed

26. ↵
    1. Duron JJ,
    2. Duron E,
    3. Dugue T,
    4. et al

    . Risk factors for mortality in major digestive surgery in the elderly: a multicenter prospective study. Ann Surg 2011;254:375–82.

    OpenUrlCrossRefPubMed