| title | subtitle | author | date | output | ||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Association between leadership commitment and professional development at NASA (2020): unweighted sex-adjusted stratified analysis |
DOCUMENT: SAR-2022-011-GJ-v01 |
**From:** Felipe Figueiredo **To:** techsavvy32 (fiverr.com) |
2022-02-19 |
|
\newpage
Association between leadership commitment and professional development at NASA (2020): unweighted sex-adjusted stratified analysis
Document version
| Version | Alterations |
|---|---|
| 01 | Initial version |
- CI: confidence interval
- FEVS: Federal Employee Viewpoint Survey
- OPM: U.S. Office of Personnel Management
- OR: odds ratio
The Federal Employee Viewpoint Survey (FEVS) addressed leadership commitment, professional development, and telework satisfaction while accounting for gender (OPM, 2020). This analysis addresses a subset of the FEVS survey reflecting NASA employees.
Quantify the association between leadership commitment and employee professional development at NASA from the 2020 Federal Employee Viewpoint Survey, disregarding the FEVS survey weights.
Raw data was collected as a census of the eligible population (OPM, 2020), and statistical weighting was applied at the data collection to adjust for non-responses in in the census attempt. These survey weights
allow for the estimation of the association under study in the source population, but this analysis will not adjust the estimates with the survey design effects (see Observations).
The raw data was filtered to reflect only NASA survey respondents (where agency code equals NN).
The original data base had
8
variables collected on
10588
observations.
Survey questions measured responses in a 5-point Likert scale between 1 (strongly disagree) and 5 (strongly agree). Some questions offered the option to choose "X" (Don't know) as the answer. These unknown answers were considered non-answers and treated as missing values.
This analysis will focus on two questions from the FEVS survey, where the main interest is employee satisfaction
(q1 -- I am given a real opportunity to improve my skills in my organization)
as the dependent variable and leadership commitment
(q21 -- Supervisors in my work unit support employee development)
as the independent variable.
As per the data cleaning process, the dependent variable was renamed to dv and the independent variable was renamed to iv in the analytical dataset.
Additionally, in order to calculate the OR the survey responses were categorized as binary responses, where agreement was aggregated from the "agree" and "strongly agree" responses, in variables dv2 and iv2.
The study includes respondents that have answered the survey in agreement (4 or 5) or disagreement (1 or 2) to any of the questions. The participants that answered 3 (Neither agree nor disagree) will be excluded.
For this case-control study the exposures were defined as high levels of leadership commitment. The group exposed to high levels of leadership commitment is defined as the survey respondent having expressed agreement with the question (q21 -- Supervisors in my work unit support employee development). Surveys in disagreement with this question were classified as the group not exposed to commitment.
The cases and controls are defined in the primary outcome (q1 -- I am given a real opportunity to improve my skills in my organization), described below.
Odds of participants that perceive opportunities of employee professional development at NASA from the 2020 Federal Employee Viewpoint Survey.
The association between exposures and the outcome will be stratified by the sex of survey respondents.
Survey characteristics will be described as counts and proportions (%). The distributions of participants' characteristics will be summarized in tables and visualized in exploratory plots. Differences in distribution of categorical variables will be assessed with the chi-square test without Yates correction. The OR will be used as a measure of effect of the independent variable on the dependent variable.
The stratification by sex will be used to assess if the effect changes across male and female strata. As a rule of thumb, a minimum change of 20% in the OR will be accepted before concluding that there is an interaction between sex and the association. The homogeneity of the OR across strata will be assessed with the Cochran-Mantel-Haenszel test.
All evaluations will be performed as complete case analyses.
All analyses will be performed using the significance level of 5%.
This analysis was performed using statistical software R version 4.1.2.
The raw database contained 10588 surveys, of which 1183 incomplete cases were removed. Of the complete cases 1920 participants did not agree nor disagree with at least one of the survey questions. These did not meet the inclusion criteria and were also removed from the analysis. After the inclusion and exclusion criteria were applied the final number of surveys in the analysis is 7485. Table 1 shows the characteristics of the surveys included in the final analysis.
Table: Table 1 Characteristics of the study population.
| Characteristic | N = 7,485 |
|---|---|
| Sex, n (%) | |
| Male | 4,764 (64%) |
| Female | 2,721 (36%) |
| I am given a real opportunity to improve my skills in my organization., n (%) | |
| Strongly disagree | 73 (1.0%) |
| Disagree | 166 (2.2%) |
| Agree | 2,948 (39%) |
| Strongly Agree | 4,298 (57%) |
| Supervisors in my work unit support employee development., n (%) | |
| Strongly disagree | 67 (0.9%) |
| Disagree | 81 (1.1%) |
| Agree | 1,986 (27%) |
| Strongly Agree | 5,351 (71%) |
One third of the employees at NASA are women (36.4%, Table 1). Both survey questions addressed in this study showed most NASA employees demonstrated high levels of satisfaction when the survey was conducted. The proportion of employees that agree or strongly agree with the satisfaction question (q1 -- I am given a real opportunity to improve my skills in my organization) was 96.8%. The proportion of employees that agree or strongly agree with leadership commitment (q21 -- Supervisors in my work unit support employee development) was 98.0%. Figure 1 shows how both sexes answered these questions.
**Figure 1** Dichotomized survey responses, by sex.
Men and women tended to answer the satisfaction question at the same rates in 2020. Of the 7246 NASA employees that are in agreement with the satisfaction question, 36.3% were women, and of the 239 employees that are in disagreement, women were 38.9% (Figure 1).
A binary categorization was performed between the two survey responses, where "Agreement" aggregates all answers "Agree" and "Strongly agree". In order to compare how the responses to the leadership commitment and employee development relate to each other, a cross-tabulation of the dichotomized answers from each study participant is shown in Table 2. This cross tabulation shows that the largest proportion of survey respondents are in agreement, simultaneously, to both questions.
The overall (unadjusted) effect of the association was significant (OR: 183.33, 95% CI: [121.86, 282.44], p<0.001). This means that when survey respondents perceive leadership commitment, they are 183 times as likely to report employee development as participants that do not perceive such commitment from leadership. Alternatively this translates to a 182-fold increase in the group that perceives high levels of leadership commitment when compared to the group that does not perceive such levels of commitment from their management.
In order to control for a possible confounder between the association and the sex of the study participant, the analysis was stratified by sex. The Mantel-Haenszel adjustment for the OR was similar to the overall unadjusted effect estimate (adjusted OR: 180.62, 95% CI: [118.65, 274.96], p<0.001). For comparison, the relative difference between the overall unadjusted effect of association and the adjusted effect is on the order of 1.5%, so we may rule out confounding between sex and this association. If there was no interaction between sex and the association between leadership commitment and satisfaction it would be sufficient to report the unadjusted OR.
Table: Table 2 Cross tabulation of dichotomized responses, overall and by sex of survey respondents.
| Group | Characteristic | Disagreement | Agreement | Total |
|---|---|---|---|---|
| Overall | Supervisors in my work unit support employee development., n | |||
| Disagreement | 113 | 35 | 148 | |
| Agreement | 126 | 7,211 | 7,337 | |
| Total, n | 239 | 7,246 | 7,485 | |
| Males | Supervisors in my work unit support employee development., n | |||
| Disagreement | 69 | 15 | 84 | |
| Agreement | 77 | 4,603 | 4,680 | |
| Total, n | 146 | 4,618 | 4,764 | |
| Females | Supervisors in my work unit support employee development., n | |||
| Disagreement | 44 | 20 | 64 | |
| Agreement | 49 | 2,608 | 2,657 | |
| Total, n | 93 | 2,628 | 2,721 |
In order to assess whether or not there is an interaction between sex and the association under study we can compare the difference between the effects observed in each sex strata. The effect sizes for men (OR: 270.42, 95% CI: [151.84, 505.11], p<0.001) were different to the effect of women (OR: 115.35, 95% CI: [64.12, 215.70], p<0.001). For comparison, the relative difference between the effect of association on males and females is on the order of 134.4%, so we cannot rule out interaction between sex and the association in this study. It is recommended to report stratum-specific effects of association in this context.
In summary, there is evidence of interaction with sex. This means that, when they perceive high levels of leadership commitment, both men and women appear to show different levels of telework satisfaction when compared to the group that does not perceive such levels of commitment from their management. In this context it is recommended to report the estimates of effect for both sexes as the result of the analysis. This recommendation also accounts for confounding by sex.
Sometimes some sub-populations of the target population can be hard to find or are scarce. Complex survey designs allow for the study estimations be performed with over-sampling of these sub-populations. The FEVS survey used statistical weighting to adjust for the sampling design effects of the survey. By involving them in the previous analysis it effectively simulated a census of the NASA employees perception of the workplace culture (see associated analysis SAR-2022-007-GJ-v02). Its large sample provided high levels of statistical power to detect even small differences in proportions. This is one reason that most p-values could be reasonably expected to be significant in that analysis.
This unweighted version of the analysis does not consider the survey weights, and as such, does not take into account the design effects of the FEVS survey. Although this choice allows for simpler methods to be used it may also lead to bias in the estimated odds ratios.
Note that the previous analysis also used a different inclusion criteria where all survey responses were used. The results from this analysis might not be directly comparable to that one given this difference in methodological choices.
One third of NASA employees in this study are women. Both men and women tended to perceive similar rates of opportunities at the workplace in 2020.
There is a positive association between leadership commitment and employee development where employees that perceive high levels of commitment also perceive more opportunities for development, when compared with employees that do not perceive high leadership commitment. Men who are exposed to high levels of commitment in the workplace environment also perceive 270 times as much opportunities for development, when compared to men working under less committed leadership. Similarly, women working under highly committed leadership also perceive 115 times as much opportunities for development, when compared to women under less committed leadership.
- SAP-2022-011-GJ-v01 -- Analytical Plan for Association between leadership commitment and professional development at NASA (2020): unweighted sex-adjusted stratified analysis
- SAR-2022-008-GJ-v02 -- Association between leadership commitment and professional development at NASA (2020): weighted sex-adjusted stratified analysis
- OPM (2020). 2020 Federal Employee Viewpoint Survey -- Technical report (https://www.opm.gov/fevs/reports/technical-reports/).
- Zarin DA, et al. The ClinicalTrials.gov results database -- update and key issues. N Engl J Med 2011;364:852-60 (https://doi.org/10.1056/NEJMsa1012065).
- Gamble C, et al. Guidelines for the Content of Statistical Analysis Plans in Clinical Trials. JAMA. 2017;318(23):2337–2343 (https://doi.org/10.1001/jama.2017.18556).
**Figure A1** Raw survey responses, by sex.
This analysis is part of a larger project and is supported by other analyses, linked below.
Association between leadership commitment and professional development at NASA (2020): weighted sex-adjusted stratified analysis
https://philsf-biostat.github.io/SAR-2022-008-GJ/
Association between leadership commitment and telework satisfaction at NASA (2020): weighted sex-adjusted stratified analysis
https://philsf-biostat.github.io/SAR-2022-007-GJ/
Association between leadership commitment and telework satisfaction at NASA (2020): unweighted sex-adjusted stratified analysis
https://philsf-biostat.github.io/SAR-2022-010-GJ/
Both this document and the corresponding analytical plan (SAP-2022-011-GJ-v01) can be downloaded in the following address:
https://philsf-biostat.github.io/SAR-2022-011-GJ/
Due to confidentiality the data-set used in this analysis cannot be shared online in the public version of this report.
| id | dsex | dv | iv | dv2 | iv2 |
|---|---|---|---|---|---|
| 1 | |||||
| 2 | |||||
| 3 | |||||
| ... | |||||
| N |
Table: Table A1 Analytical dataset structure