Highlights
- Companion animal veterinarians had lower disease-reporting knowledge.
- Large animal veterinarians had higher knowledge of biosecurity practices.
- Discrepancy between veterinarians’ attitudes and practices was identified.
- Training improved disease risk perception, reporting, and biosecurity practices.
Abstract
Veterinarians play an essential role in safeguarding and promoting animal and human health by timely reporting of notifiable diseases to animal and public health agencies and by educating animal owners on effective disease prevention measures. Moreover, clinical veterinarians can prevent the transmission and spread of zoonotic diseases by adopting effective biosecurity practices in their clinics.
An online questionnaire was administered between October and November 2021 to veterinarians registered with the Illinois State Veterinary Medical Association. Veterinarians were surveyed on their disease risk perception, biosecurity practices, and disease reporting knowledge. In total, 104 veterinarians (64 % females and 46 % males) completed the questionnaire, of whom 88 % were veterinarians working in clinical practice (88 % companion animals and 12 % bovine or swine), while 12 % were employed in non-clinical settings. The disease-reporting knowledge score was higher for veterinarians with biosecurity training (IRR: 1.35; 95 % CI: 1.47–1.75). Compared to large animal veterinarians, companion animal veterinarians had lower odds of having biosecurity training (OR=0.68; 95 % CI=0.02–0.28) and were less familiar with current biosecurity guidelines (OR=0.12; 95 % CI = 0.03–0.51). Veterinarians familiar with biosecurity guidelines had a higher probability (OR=4.4; 95 % CI: 1.21–16.28) of perceiving biosecurity practices as practical while working with animals. Conversely, veterinarians who perceived that they could transmit diseases to animals had lower odds (OR: 0.42; 95 % CI: 0.20–0.91) of wearing protective clothing while dealing with confirmed cases of zoonotic diseases.
Based on our study results, a gap in disease-reporting knowledge was identified among companion animal veterinarians. Biosecurity training improved the disease-reporting knowledge of veterinarians, suggesting that providing continuing education for veterinarians would be beneficial to disease reporting. A disconnect between disease risk perception and biosecurity practices was identified and further studies are needed to understand this discrepancy to design effective education programs.
Abbreviations
Keywords
1. Introduction
The emergence and spread of infectious diseases impact livestock farms, reducing their productivity and causing economic setbacks in the agricultural economy (Smith et al., 2019). In the United States of America (US), foreign animal diseases (FAD), such as African swine fever, pose a constant threat to swine populations (Brown et al., 2021, Brown and Bevins, 2018). The risk of FAD introduction underscores the critical need to develop effective biosecurity protocols to prevent and control the spread of infectious diseases on livestock farms and veterinary clinics (Morley, 2002). Successful disease prevention and management can be achieved by adopting stringent biosecurity practices, disease monitoring and surveillance, early disease reporting, and rapid outbreak response. (Ellwanger et al., 2019, Moore and Lund, 2009).
Veterinarians play a critical role in improving animal health and welfare by treating sick animals and educating animal owners on effective disease prevention practices (Alarcon et al., 2021, Renault et al., 2018). Animal owners also consider them a trusted source of information (Alarcon et al., 2021, Kemal, 2014); therefore, evaluating their perceptions and practices related to biosecurity is crucial. Veterinarians’ attitudes toward biosecurity are belief-driven, which can affect their practices and may influence the advice they provide to their clientele. Consequently, interventions for behavior modifications related to biosecurity should consider veterinarians’ perceptions (Ajzen, 1991, Heath et al., 2015).
In the US, veterinarians have a central role in reporting foreign (e.g., African swine fever) and zoonotic (e.g., rabies) diseases to animal and/or public health authorities (Allen, 2012, Trevejo, 2009). Timely disease reporting is vital for health agencies to start a rapid response to curb disease spread. However, studies have reported a gap in disease-reporting knowledge and unfamiliarity with disease notification pathways among US veterinarians (Venkat et al., 2019).
Adopting effective infection prevention and control practices by veterinarians and their staff when working with infected animals prevents zoonotic disease transmission (Lipton et al., 2008, Weese et al., 2002). However, studies have indicated low-level of compliance with biosecurity practices among veterinarians (Dowd et al., 2013, Hardefeldt et al., 2018, Lipton et al., 2008, Venkat et al., 2019, Wright et al., 2008) and described low standards of hand hygiene and inconsistent use of personal protective equipment (Hardefeldt et al., 2018, Noremark and Sternberg-Lewerin, 2014, Sahlstrom et al., 2014, Venkat et al., 2019, Wright et al., 2008).
Considering all the issues presented before, our study objectives are to assess Illinois veterinarians’ disease risk perception and perceived relevance of biosecurity practices in infection prevention and control, evaluate their knowledge of disease reporting, and describe their disease prevention and control practices in clinical settings.
2. Methods
2.1. Study design
An online cross-sectional survey was administered between September 2021 and November 2021 to veterinarians registered (n=1391) with the Illinois State Veterinary Medical Association (ISVMA). Each veterinarian received a personalized email with a survey link at the beginning of the study. In addition, two reminder e-mails were sent, and the survey link was shared once in the weekly ISVMA electronic newsletter.
2.2. Development of survey instrument
The structured questionnaire was designed to collect information from the Illinois veterinarians on their disease reporting knowledge, their perception and attitude toward disease risk, the role of biosecurity in infection prevention and control, and biosecurity practices employed by clinical veterinary practitioners (Supplementary File 1).
The interactive survey contained 45–47 open- and close-ended questions. The range of questions implied a logical display attribute to show only relevant questions to veterinarians based on their practice. The questionnaire was divided into six sections: general demographic characteristics (7 questions), disease risk perception and disease reporting knowledge (10 questions), disease investigation and reporting (3 questions), demographic characteristics of clinical veterinary practitioners (5 questions), biosecurity practices of clinical veterinary practitioners tailored to their type of practice (18–20 questions), and sources of updated biosecurity information (2 questions). The questionnaire was forked based on the type of practice using the Qualtrics software logic attribute. In addition, we tailored the practice-related questions to the practice type for each category of clinical veterinary practitioner (companion animal veterinarian, swine veterinarian, bovine veterinarian, equine veterinarian, and mixed animal veterinarian). The questionnaire was reviewed by subject matter experts and was pretested by 24 graduate students from the College of Veterinary Medicine, University of Illinois, Urbana-Champaign. The majority of the graduate students enrolled in pretesting were foreign-trained veterinarians. Reviewing and pretesting aided in evaluating the validity of the survey instrument and identifying errors in the survey flow. The questionnaire was revised after the review and pretesting to draft the final version used for the study. The survey was designed to take an estimated 10 min to complete.
2.3. Data analysis
The survey data were imported into Excel (.xlsx) and comma-separated values (.csv) formats from QualtricsXM data files. Microsoft Excel (version 16.70, Microsoft Corporation, Redmond, Washington, USA) was used for data management and coding. R Studio (Version 1.4.1106 2009–2021 RStudio, PBC) was used for data visualization, and STATA Intercooled software (Version 17, Stata Corporation, College Station, TX) was used for statistical analysis.
2.3.1. Descriptive analysis
Summary statistics were conducted for variables corresponding to respondents’ demographic and practice characteristics and presented as frequencies, proportions, means (for continuous variables), and medians (for count variables). Some categorical variables were recategorized based on their distribution using Lowess smoothing. The variable representing veterinarians’ graduation year for the primary degree in Veterinary Medicine (eg. DVM, BVSc, BVM) was recategorized into three categories (1965–1984, 1985–2004, and 2004–2021), years of experience in clinical practice was recategorized into two categories (less than or equal to 25 years and more than 25 years) and having an additional degree post-primary veterinary medicine degree was categorized into a binary variable (“Yes” or “No”).
2.3.2. Development of a scoring method for disease risk perception, knowledge of disease reporting, and biosecurity practices
2.3.2.1. Disease risk perception scores
The survey assessed veterinarians’ foreign (a disease that is not currently found in the United States) and infectious disease risk perceptions and attitudes toward the importance of infection prevention and control measures. In total, there were seven perception assessment questions in the questionnaire. All questions were assessed on a Likert scale and depicted in a Likert graph using RStudio packages: Likert (Jason Bryer, 2016), HH (Heiberger & Holland, 2015) and ggplot2 (Wickham, Hadley, 2016). Summary statistics were used, and mean scores and standard deviations were computed for all the Likert scale questions (Alarcon et al., 2021, Kemal, 2014).
2.3.2.2. Disease reporting knowledge scores
For a list of diseases, including FAD, notifiable diseases, endemic diseases, and diseases of public health significance, veterinarians were asked to indicate to which listed animal and public health agencies they should report a suspected case of a given disease. The options “Will treat the animal and not report to anyone” and “I don’t know” were also provided. All questions were multiple-answer, as certain diseases need to be reported to multiple agencies. A score of “1” was assigned for even a single correct option from the listed agencies and “0” for the incorrect responses. Respondents choosing “I don’t know” were given a score of “0”. The scores for all seven questions were summed, and a score was assigned to each veterinarian. A higher score indicated better disease-reporting knowledge. Summary statistics and cross-tabulated scores with demographic characteristics were made to assess the differences in the knowledge scores across veterinarians in different work settings.
2.3.2.3. Biosecurity practice scores
Veterinarians were asked to report their biosecurity practices related to infection prevention and control. The practice questions were tailored to their practice type (companion animal, swine, cattle, equine, mixed). Practice type was recategorized into two categories: small animal veterinarians (treating predominantly companion animals) and large animal veterinarians (treating swine, cattle, equine, or mixed species) for the analysis. Each practice question response was recorded on a 4-point Likert scale (always, sometimes, rarely, never). As biosecurity practices for infection prevention and control are required to be followed consistently, the responses were recategorized as a binary variable “Yes” for “always” and “No” for “sometimes, rarely, never”. The variables were scored as 1 for “Yes” and 0 for “No”. A total practice score was computed for each veterinarian by summing scores for all practice-related questions.
2.3.3. Multiple correspondence analysis
Multiple correspondence analysis (MCA) is a categorical variable counterpart of principal component analysis (Abdi and Williams, 2010). The MCA follows dimension reduction by converting the correlation between multiple categorical variables into a two-dimensional plot. It helps identify potential links between qualitative and/or quantitative variables (Husson et al., 2017). MCA was conducted on the survey data to demonstrate potential relationships between veterinarians’ perceptions, biosecurity knowledge, infection prevention and control practices, and demographic characteristics. A two-dimensional symmetric map was constructed to illustrate the two-way interaction solution of the variable categories. After the MCA, hierarchical clustering on principle components (HCPC) was applied using Euclidean distances to identify clusters (Argüelles et al., 2014).
The MCA was conducted using Burt’s method, and the HCPC was performed using Ward’s criteria in RStudio. The FactoMineR (Husson et al., 2016), factoextra (Kassambara and Mundt, 2017), and tidyverse (Wickham and Wickham, 2017) R packages were used to conduct MCA and HCPC to visualize their results.
2.3.4. Regression analysis
2.3.4.1. Logistic regression models
Multivariable logistic regression models were constructed using the forward selection technique (Freund et al., 2006). In the multivariable models, variables with a relaxed p-value of <0.20 at the univariable stage were included. The final multivariable model included variables with a p-value ≤0.05 on the Wald χ2 test. As the measure of effects, odds ratios (OR), their 95 % confidence intervals, and p values were calculated.
The first multivariable model assessed the association between veterinarians’ perception of disease risk and the importance of biosecurity in infection prevention and control as the outcome variable (“1” for positive perception; “0” for neutral or negative perception) and various predictor variables (demographic characteristics, having biosecurity training, familiarity with biosecurity guidelines, and participation in FAD investigations).
The second multivariable model evaluated associations between biosecurity practices as the outcome variables and demographic characteristics as predictor variables.
In addition, several univariable models were built to test associations between veterinarians’ perceptions and practices using logistic regression, considering each practice (yes or no) as an outcome variable and each answer to the perception-based question as the predictor variable.
2.3.4.2. Poisson regression models
Using the forward selection technique, we constructed two multivariable Poisson regression models (Lovett and Flowerdew, 1989). The first model had the knowledge score as the outcome variable and demographic characteristics as predictor variables. The second model included the practice score as the outcome variable and demographic characteristics as predictor variables. Variables identified at the univariable stage with a relaxed p-value of <0.20 were included in the subsequent model. The final multivariable model included variables with a p-value ≤0.05. An incidence rate ratio (IRR), 95 % confidence intervals, and p-value were calculated for each significant predictor.
2.4. Ethics
The Institutional Review Board (IRB) of the University of Illinois at Urbana Champaign approved the study procedure (IRB #21974). We complied with the set guidelines for study methodology and data collection. Respondents’ participation in the study and completion of the survey constituted implied consent.
3. Results
3.1. Demographic characteristics of veterinarians
We received 121 responses for the survey, of which 119 responses were included in the study. The response rate for the survey was 8.6 %. Table 1 describes the demographic characteristics of the respondents. Within the profession category, veterinarians who indicated their profession as “veterinarians with another status” (n=3) reported working in a “nonpracticing administrative position, in the industry or are retired”. Sixteen veterinarians had a graduate degree, eight had a board certification, and three respondents had both as additional qualifications. Among the veterinarians with biosecurity training (n=24), sixteen received their training within the last five years, and six received it more than five years ago. The biosecurity training was reported to be obtained through a combination of “Continuing Education credits” (n=9), “webinars” (n=9), “in-person training” (n=10), “USDA training courses” (n=3), and “graduate certification and accreditation” (n=2).
Table 1. Demographic characteristics of Illinois veterinarians.
| Demographics | Frequency (Percentage) |
|---|---|
| Profession (n=119) | |
| Academia | 7 (6.2) |
| Clinical practice | 105 (88) |
| Government official | 3 (2.5) |
| Animal welfare sector | 1 (0.8) |
| Veterinarians with another status | 3 (2.5) |
| Gender of respondents (n=117) | |
| Male | 42 (36) |
| Female | 75 (64) |
| Veterinary school graduation year (n=119) | |
| 1965–1984 | 24 (20) |
| 1984–2004 | 56 (47) |
| 2005–2021 | 39 (33) |
| Graduate Degree/Board certification (n=117) | |
| Yes | 21 (18) |
| No | 97 (82) |
| Received Biosecurity Training (n=117) | |
| Yes | 24 (20) |
| No | 93 (80) |
3.2. Veterinarians’ perception of biosecurity measures
3.2.1. Descriptive analysis
Veterinarians were asked to indicate their responses on a Likert scale for perception-based questions. The questions inquired about veterinarians’ perceived importance of biosecurity measures, the likelihood of disease transmission and outbreak occurrence, and the practicality of adopting biosecurity measures. The proportion of responses to each category for all the questions is illustrated in Fig. 1.
Fig. 1. Likert scale responses to biosecurity perception-based questions: a higher score of 4–5 (agree/ strongly agree) indicates favorable perception, a lower score of 1–2 (disagree/ strongly disagree) indicates unfavorable perception and a middle score of 3 indicates neutral stand (neither agree nor disagree). Perception_1: Importance of biosecurity practices for the prevention and control of FAD. Perception_2: Importance of biosecurity practices for the prevention and control of infectious diseases, Perception_3: Importance of developing disease prevention and control plans to prepare for a FAD outbreak, Perception_4: Importance of disease surveillance and testing for the detection and prevention of infectious diseases, Perception_5: Likelihood of the occurrence of a FAD outbreak in the US mainland in the next three years, Perception_6: Likelihood of a veterinarian transmitting an infectious disease from one animal to another, Perception_7: Practicality of following effective biosecurity measures while handling animals in day-to-day practice.
The mean and standard deviation of the perception score of each perception-based question for all the survey respondents are presented in Table 2. The overall mean perception score of the survey was 4.33, corresponding to a positive/favorable perception of disease risk and biosecurity practices.
Table 2. Description of the perception-based question scores.
| Perception Questions | Responses | Mean*1 | SD |
|---|---|---|---|
| How important do you consider biosecurity practices for the prevention and control of foreign animal diseases (FADs)? | 113 | 4.81 | 4.30 |
| How important do you consider biosecurity practices for the prevention and control of infectious diseases? | 112 | 4.84 | 4.33 |
| How important is developing disease prevention and control plans to prepare for a foreign animal disease (FAD) outbreak? | 113 | 4.65 | 2.85 |
| How important do you consider disease surveillance and testing for the detection and prevention of infectious diseases? | 113 | 4.65 | 4.15 |
| What do you think is the likelihood of the occurrence of a foreign animal disease (FAD) outbreak in the US mainland in the next 3 years? | 105 | 3.79 | 4.15 |
| What is the likelihood of a veterinarian transmitting an infectious disease from one animal to another? | 104 | 3.61 | 3.36 |
| How practical is it to follow effective biosecurity measures while handling animals in day-to-day practice? | 104 | 3.97 | 3.23 |
- *
-
The range of interpreting the Likert scale mean score: 1.0–2.4 (Negative attitude), 2.5–3.4 (Neutral attitude), and 3.5–5.0 (Positive attitude). 1Mean= sum(number of responses for a score 1*1, number of responses for a score 2*2, number of responses for a score 3*3, number of responses for a score 4*4, number of responses for a score 5*5)/total number of responses
Veterinarians were also asked about their familiarity with current guidelines and practices for preventing and controlling foreign animal disease (FAD) outbreaks. Fifty-nine percent (n=67) responded that they were “extremely or moderately familiar”, 37 % (n=42) responded that they were “somewhat or slightly familiar”, and 4 % (n=4) responded that they were “not at all familiar”.
3.2.2. Veterinarians’ perceptions of disease risk and the importance of biosecurity practices: multiple correspondence analysis and hierarchical clustering on principal components (MCA and HCPC)
The MCA solution for the veterinarians’ perception of disease risk and the importance of biosecurity measures are illustrated in Fig. 2.
Fig. 2. Biosecurity and disease risk perception: Multiple correspondence analysis solution (a) Active variables1 in the MCA solution with gradient colored according to their quality of representation (squared cosine) on the map with supplementary variables (Green) (b) Distribution of the veterinarians used in the MCA solution gradient colored according to their quality of representation (squared cosine) on the map. MCA factor map with its 95 % confidence ellipse within the two MCA dimensions represented according to their (c) gender and (d) Vet school graduation year. The individuals are colored according to the variable category they represent. Active Variables1: P1: Importance of biosecurity practices for the prevention and control of FAD, P2: Importance of biosecurity practices for the prevention and control of infectious diseases, P3: Importance of developing disease prevention and control plans to prepare for an FAD outbreak, P4: Importance of disease surveillance and testing for the detection and prevention of infectious diseases, P5: Likelihood of the occurrence of an FAD outbreak in the US mainland in the next 3 years, P6: Likelihood of a veterinarian transmitting an infectious disease from one animal to another, P7: Practicality of following effective biosecurity measures while handling animals in day-to-day practice (**P1_++, P1_+, P1, P1_-, P1_– indicates Likert scale score 5,4,3,2,1, respectively. Likewise, for P2, P3, P4, P5, P6, P7), Grad_yr: Vet School Graduation Year categories, Y65_84 (1965–1984); Y85_04 (1985–2004), Gen: Gender category, M (male); F(Female) Supplementary variables: Veterinarians’ Profession: Clin_V: Clinical Veterinary Practitioner, Aca_V: Academic Veterinarian, Ani_V: Veterinarian in Academia, Gov_V: Veterinarian in Government, Oth_V: Others Category Biosecurity training Status: Bio_y: Yes, Bio_n: No.
The two-dimensional MCA solution recovered 17 dimensions to explain the total variability of the data (Supplementary File 2: S_Table 1; S_Fig. 1). The first two dimensions explained 26.3 % of the variance in the data (Dim1: 16.1 %, Dim2: 10.2 %). The variable categories, including gender, Vet school graduation year, and perception-based questions about the importance of biosecurity measures (P1, P2, P3, P4), were well represented in the first two dimensions (Supplementary File 2: S_Fig. 2). However, the P5, P6, and P7 variable categories required the inclusion of higher dimensions for quality representation. Therefore, their association should be interpreted with caution. We relied on the first two MCA dimensions to investigate the variable relationships to illustrate an interpretable graphic representation of the MCA results.
In Fig. 2, Dimension 1 was primarily influenced by perception questions about the importance of biosecurity measures (P1: 0.585; P2: 0.541; P3: 0.488; P4: 0.437), whereas Dimension 2 was influenced by gender (0.657) and graduation year (0.399). Non-clinical veterinarians (e.g., government, academic) with biosecurity training were closely associated with positive perceptions of the importance of biosecurity measure variable categories (P1_++, P2_++, P3_++, P4_++) (Fig. 2a). In Fig. 2b, veterinarians are represented on a color gradient based on their contribution to the MCA solution. Veterinarians who were close together shared similar perceptions of disease risk and the importance of biosecurity measures. Fig. 2c and d represent the MCA factor map with its 95 % confidence ellipse. Here, the non-superimposition of the confidence ellipses implies that the categories within a variable are considered different (Husson et al., 2017). Our results suggest that the perceptions of males were different from those of females (Fig. 2c), and the perceptions of graduates of Y_85–04 were different from those of graduates of Y_65–84 and Y_05–21 (Fig. 2d). Moreover, the male gender and vet school graduation year 1985–2004 presented similar confidence ellipse profiles on the MCA factor map, indicating the relatedness of the two variable categories (Fig. 2c and d).
The results from the final MCA solution (ten active variables; two supplementary variables) were subsequently used to perform HCPC. We included all the components in the hierarchical clustering to retain the maximum possible information. The partitioning of the tree clustering from the HCA presented three clusters (Supplementary File 2: S_Fig. 3).
3.2.3. Logistic regression analysis on veterinarians’ disease risk perceptions and biosecurity practices
The univariable logistic regression analysis results revealed that veterinarians with biosecurity training had 3.33 times higher odds (95 % CI: 1.10–10.09; p value=0.03) of perceiving the likelihood of an FAD outbreak in the next three years in the US mainland. The veterinarians familiar with the current biosecurity guidelines had higher odds (OR=4.4; 95 % CI: 1.21–16.28; p=0.02) of perceiving biosecurity practices as practical while handling animals in day-to-day practice. The associations with other variables were not significant.
3.3. Veterinarians’ disease reporting knowledge
3.3.1. Descriptive analysis
The summary statistics for the disease reporting knowledge for all veterinarians, with comparisons by work setting (clinical vs. non-clinical) and clinical practice type (companion animal vs. large animal practitioners), are reported in Table 3.
Table 3. Summary statistics for the disease reporting knowledge score for all veterinarians (n=99).
| Empty Cell | All | Work Setting | Clinical Practice Type | ||
|---|---|---|---|---|---|
| Non-Clinical | Clinical | Companion Animal | Large Animal | ||
| Number | 99 | 11 | 88 | 77 | 11 |
| Mean | 3.28 | 4.18 | 3.17 | 3.00 | 4.36 |
| S.D. | 1.42 | 1.40 | 1.39 | 1.33 | 1.29 |
| Range | 0–7 | 2–7 | 0–6 | 0–5 | 2–6 |
| Median | 3 | 4 | 3 | 3 | 5 |
In addition to specific disease reporting, veterinarians were asked in what situation they would report a FAD and a state/national reportable disease; 15 veterinarians indicated they would report a suspected case, 17 veterinarians said they would report a confirmed case, and 69 veterinarians indicated they would report both suspected and confirmed cases.
Seven veterinarians reported being a part of a FAD (suspected/diagnosed) investigation in their careers as clinical veterinary practitioners. Eighteen veterinarians suspected/diagnosed a notifiable animal disease in Illinois in the past three years. Of these, eleven reported it to either one or a combination of indicated agencies: (State Animal Health Official (SAHO), United States Department of Agriculture (USDA) – Animal and Plant Health Inspection Service (APHIS) – Area veterinarian-in-charge (AVIC), State Public Health Agency. Two veterinarians did not report the encountered case of suspected/diagnosed notifiable animal disease to any of the health authorities. One of these two respondents cited no confirmatory diagnosis (“the case turned out to not have rabies”) as the reason for not reporting.
3.3.2. Veterinarians’ disease reporting knowledge: multiple correspondence analysis and hierarchical clustering on principal components (MCA and HCPC)
Fig. 3 illustrates the MCA solution for the disease-reporting knowledge-based questions.
Fig. 3. Disease Reporting: Multiple Correspondence Analysis solution (a) Active variables1 in the MCA solution with gradient colored according to their quality of representation (squared cosine) on the map with supplementary variables (Green) (b) Distribution of the veterinarians used in the MCA solution gradient colored according to their quality of representation (squared cosine) on the map. MCA factor map with its 95 % confidence ellipse within the two MCA dimensions represented according to their (c) gender and (d) biosecurity training status. The individuals are colored according to the variable category they represent. Active Variables1: Rab_R: Rabies reported to the correct agency, Rab_W: Rabies reported to the incorrect agency, ASF_R: ASF reported to the correct agency, ASF_W: ASF reported to the incorrect agency, FMD_R: FMD reported to the correct agency, FMD_W: FMD reported to the incorrect agency, PRRS_R: PRRS reported to the correct agency, PRRS_W: PRRS reported to the incorrect agency, EI_R: EI reported to the correct agency, EI_W: EI reported to the incorrect agency, Lepto_R: Leptospirosis reported to the correct agency, Lepto_W: Leptospirosis reported to the incorrect agency, Salmo_R: Salmonellosis reported to the correct agency, Salmo_W: Salmonellosis reported to the incorrect agency, Biosecurity training Status: Bio_y: Yes, Bio_n: No. Gen: Gender category, M (male); F(Female) Supplementary variables: Veterinarians’ Profession: Clin_V: Clinical Veterinary Practitioner, Aca_V: Academic Veterinarian, Ani_V: Veterinarian in Academia, Gov_V: Veterinarian in Government, Oth_V: Others Category, Fam_guide: Familiarity with the biosecurity guidelines, Familiar_y: Yes; Familiar_No,.
The two-dimensional MCA solution recovered 9 dimensions to explain the total variability of the data (Supplementary File 2: S_Table 2; S_Fig. 4). The first two dimensions explained a total of 45.3 % of the variance in the data (Dim1: 26.8 %, Dim2: 18.5 %). Most of the variable categories used in the MCA were well represented in the first two dimensions (Supplementary File 2: S_Fig. 5). Therefore, we presented and used the first two MCA dimensions to investigate the variable relationship. In Fig. 3, Dimension 1 is primarily influenced by gender (0.257), biosecurity training status (0.230), and reporting of FMD (0.486), equine influenza, EI (0.396), porcine reproductive and respiratory syndrome (PRRS) (0.465), and ASF (0.485), whereas Dimension 2 was influenced by reporting of leptospirosis (0.621) and salmonellosis (0.638). Reporting of rabies (Rab_R; Rab_W) had a low squared cosine value corresponding to a lower quality of representation in the first two dimensions and would require higher dimensions for better representation (SFig_4). Therefore, the association of reporting of rabies (Rab_R) should be interpreted with caution. Male veterinarians, having biosecurity training and familiarity with biosecurity guidelines were closely linked with the correct responses to reporting FMD, ASF, PRRS, and EI (Fig. 3a). In Fig. 3b, the proximity between the veterinarians (n=83) used in the MCA solution for disease reporting knowledge indicates a similar response profile. The MCA factor map with 95 % confidence intervals suggests that male reporting knowledge was different from that of females (Fig. 3c), and the disease reporting of veterinarians with biosecurity training was different from that of those without biosecurity training (Fig. 3d). The relatedness of “males” and “having biosecurity training” was evident through visual assessment of their similar confidence ellipse profiles on the MCA factor map (Fig. 3c and d). “Females” and “not having biosecurity training” had similar confidence ellipse profiles on the MCA factor map.
The results from the final MCA solution (nine active variables plus two supplementary variables) were subsequently used to perform HCPC. The partitioning of the tree clustering from the HCA presented five clusters (Supplementary File 2: S_Fig. 6).
3.3.3. Poisson regression analysis of veterinarians’ knowledge scores
The results of the univariable Poisson regression models are presented in Table 4.
Table 4. The impact of various demographic factors on veterinarians’ disease reporting knowledge scores (n=102).
| Knowledge Score | IRRa | 95 % CIb | p-value | |
|---|---|---|---|---|
| Additional degree after earning a degree in veterinary medicine | 1.13 | 0.85 | 1.49 | 0.41 |
| Have biosecurity training | 1.44 | 1.12 | 1.84 | <0.001** |
| Part of a FAD investigation | 1.11 | 0.74 | 1.67 | 0.627 |
| Familiarity with current guidelines | 1.10 | 0.98 | 1.24 | 0.108 |
| Sex | 0.74 | 0.59 | 0.92 | <0.001** |
| Professors (Academics) | Referent | |||
| Clinical Veterinarians | 0.71 | 0.49 | 1.02 | 0.07 |
| Animal welfare | 0.68 | 0.21 | 2.22 | 0.52 |
| Others | 0.90 | 0.46 | 1.76 | 0.77 |
| Graduation year (1965–1984) | Referent | |||
| 1985–2004 | 0.73 | 0.57 | 0.95 | 0.02** |
| 2005–2021 | 0.64 | 0.48 | 0.86 | <0.001** |
- a
-
Incidence rate ratio
- b
-
Confidence interval
- **
-
statistically significant at p≤0.05
For non-clinical veterinarians, the rate of the knowledge score would be expected to increase by a factor of 1.40 (95 % CI: 1.03–1.93) compared to clinical veterinarians. For large animal veterinarians, the rate of the knowledge score would be expected to increase by a factor of 1.47 (95 % CI: 1.08–2.00) compared to companion animal veterinarians.
Only two predictor variables remained significant in the multivariable model. The knowledge score rate was lower for female veterinarians than for male veterinarians (IRR: 0.76; 95 % CI: 0.61–0.96; p-value: 0.02) and higher for veterinarians with biosecurity training than for veterinarians with no biosecurity training (IRR: 1.35; 95 % CI: 1.47–1.75; p-value: 0.02).
3.4. Infection prevention and control practices of clinical veterinary practitioners
3.4.1. Descriptive analysis
The demographic characteristics of veterinarians working in clinical practice (n=105) are presented in Table 5.
Table 5. Demographic characteristics of veterinarians working in clinical practice.
| Demographics | Frequency (Percentage) |
|---|---|
| Type of practice (n=91) | |
| Companion animal | 80 (88) |
| Large animal | 11 (12) |
| Large animal veterinarians (n=11) | |
| Swine | 5 (46) |
| Bovine | 3 (27) |
| Equine | 0 (0) |
| Mixed | 3 (27) |
| Region of practice (n=91) | |
| Northeast | 34 (37) |
| North-Central | 25 (28) |
| Central | 19 (21) |
| Southern | 13 (14) |
| Years of experience in clinical practicea (n=91) | |
| ≤25 years | 45 (49) |
| >25 years | 46 (51) |
| Role in the practice (n=91) | |
| Associate Veterinarian | 42 (46) |
| Owner/Co-owner | 45 (49) |
| Others | 4 (5) |
- a
-
Indicates a categorical variable recategorized into two categories.
Clinical veterinarians who selected “others” as their role in the practice indicated working as “Chief of Staff” (n=2) and “Managing DVMs” (n=2). The companion animal veterinarians examined a mean of 348.92 clients per month (95 % CI: 278.7406–419.095). The veterinarians ranked their clientele in their clinical practices in ascending order: canines, felines, exotic animals, and avians. The median (range) number of veterinarians working with the respondents in a companion animal practice was 4 (1−19).
Among the large animal veterinarians (n=11), eight veterinarians reported providing biosecurity assessments for their clients’ farms, with a median of 22.5 (range: 10–198) biosecurity assessments conducted in the last three years. In addition, the large animal veterinarians indicated the following as the most common biosecurity problem in Illinois livestock farms: “Wean to finish biosecurity is the biggest struggle and challenge”, “People”, “Workers”, “boots/vehicles moving things on and off farms without being washed or changing shoes”, “Adding new animals to the farm”, “Slaughter pig transportation”, “Traveling to fairs and trail rides and then coming back home”, “insect and transportation contamination/migration”, and “Being brought in from another country”.
Fig. 4 describes biosecurity practices followed in clinical settings as reported by companion and large animal veterinarians.
Fig. 4. Biosecurity practices in clinical settings as reported by (a) companion animal veterinarians and (b) large animal veterinarians. “Yes” (blue) indicates that they “always” follow the practice, and “no” (red) indicates that they follow it either “sometimes, rarely, or never”.
The most adopted biosecurity practice among companion animal veterinarians was disinfecting the examination table between patients, and among large animal veterinarians, it was the disposal of medical waste. Among companion animal veterinarians, washing hands between patients was followed by 73.7 % of companion animal veterinarians, and only 45.3 % changed coveralls between sick and healthy patients.
In companion animal practice, leptospirosis in dogs was indicated as the most encountered zoonotic disease (Supplementary File 2: S_Table 3).
We asked the veterinarians if they made house calls to see patients and requested information about their biosecurity practices adopted during these visits. Thirty-six percent (n=26) of companion animal veterinarians reported making house calls to treat patients. Veterinarians reported their general order of visits to see the clients in the event of multiple house calls in a day as follows: “No specific order” (n=9), “Healthy to sick animals” (n=6), “Based on the emergency of cases” (n=6), and “Based on proximity to the location (closest to farthest) (n=2). The summary statistics for other multiple visit-related questions are presented in Supplementary File 2: S_Table 4.
We asked the companion animal veterinarians about the infection prevention-specific structures and practices in their clinics. Sixty-five percent (n=49) of companion animal veterinarians indicated having a designated examination room in their clinic for patients suspected of an infectious disease. Ninety-three percent of the companion animal veterinarians reported instructing their staff to disinfect the examination rooms daily. Thirty-three percent of the veterinarians indicated instructing their staff to disinfect the waiting area at least once a day; forty-nine percent indicated instructing their staff to disinfect the waiting area multiple times a day; and fourteen percent indicated that they instructed clinic staff to clean at least once with additional disinfection if there was an infectious or contagious disease case. The remaining four percent of companion animal veterinarians reported not having patients in the waiting area due to COVID-19-related restrictions.
Seventy-nine percent of companion animal veterinarians reported wearing personal protective equipment while treating a confirmed case of zoonotic disease, whereas only 49 % reported wearing protective covers when treating both suspected and confirmed cases of zoonotic diseases.
3.4.2. Clinical veterinarians’ biosecurity practices: multiple correspondence analysis and hierarchical clustering on principal components (MCA and HCPC)
Fig. 5 illustrates the MCA solution for the biosecurity practices of clinical veterinary practitioners.
Fig. 5. Biosecurity Practices: Multiple correspondence analysis solution (a) Active variables1 in the MCA solution with gradient colored according to their quality of representation (squared cosine) on the map with supplementary variables (Green) (b) Distribution of the veterinarians used in the MCA solution gradient colored according to their quality of representation (squared cosine) on the map. Active Variables1: HandWash: Wash hands between patients Changeglove: Change disposables (like gloves, shoe covers, etc.) between patients Changeapron: Change aprons/coveralls between patients, DisinfectTool: Disinfect examination tools (Stethoscope, thermometer, etc.) between patients DiscardWaste: Discard medical waste safely (** _Always, _NotAlways indicates following a practice always and “either sometimes or rarely or never”, respectively. Grad_yr: Vet school Graduation Year categories, Y65_84 (1965–1984); Y85_04 (1985–2004), Gen: Gender category, M (male); F(Female) Region of Practice (ROP), NE: North East IL; Cen: Central IL; NorC: North Central IL; Sou: Southern IL, Biosecurity Training Status, Bio_y: Yes; Bio_n: No and Years of experience (YOE) in clinical practice, 25+: more than 25 years; 25>: less than 25 years. Supplementary variables: Practice type: CAV: companion animal veterinarian; LAV: large animal veterinarian; Role at practice: Own_V: owner; Asso_V: associate veterinarian; Oth_V: other veterinarian.
The two-dimensional MCA solution recovered 13 dimensions to explain the total variability of the data (Supplementary File 2: S_Fig 7). The first two dimensions explained 38.5 % of the variance in the data (Dim1: 20.7 %, Dim2: 17.8 %) (Fig. 5). Except for the region of practice, all the other variable categories were well represented in the first two dimensions (Supplementary File 2: S_Table 5; S_Fig 8). Therefore, the region of practice required the inclusion of additional dimensions (Dim 3, Dim 4) for better representation. Dimension 1 is primarily influenced by veterinarians’ biosecurity training status (0.254), region of practice (0.226), hand washing practices (0.389), changing gloves practices (0.434), disinfection tools (0.300), discarding waste (0. 0.494), whereas Dimension 2 is influenced by gender (0.443), graduation year (0.761), and years of experience (0.679). Companion animal veterinarians, owing the practice, and graduates of Y_85–04 were closely linked with washing hands between patients, changing disposables (such as gloves, shoe covers, etc.) between patients, disinfecting examination tools (stethoscope, thermometer, etc.) between patients, and discarding medical waste safely (Fig. 5a). The veterinarians (n=71) with similar biosecurity practices were closer in the MCA individual factor map (Fig. 5b).
Fig. 6 illustrates the MCA factor map with its 95 % confidence ellipse of veterinarians’ demographics and biosecurity training level, where the nonoverlapping ellipse indicates differences in the responses between those subcategories.
Fig. 6. MCA factor map with its 95 % confidence ellipse: represented according to their (a) Gender (Gen), M (male); F (Female), (b) Region of practice (ROP), NE: Northeast IL; Cen: Central IL; NorC: North Central IL; Sou: Southern IL (c) Biosecurity training status, Bio_y: Yes; Bio_n: No and (d) Years of experience (YOE) in clinical practice, 25+: more than 25 years; 25>: less than 25 years. The individuals are colored according to the variable category they represent.
The practices of male veterinarians were different from those of female veterinarians (Fig. 6a). Veterinarians in northeastern Illinois adopted different practices than veterinarians in other regions (Fig. 6b). The practices differed between veterinarians with and without biosecurity training (Fig. 6c). The practices of veterinarians with more than 25 years of experience were different from those of veterinarians with less than 25 years of experience (Fig. 6d). Visual assessment of Fig. 6 revealed a relatedness and similar confidence ellipse profiles among the following variables: female, northeast Illinois region, yes for biosecurity training and more than 25 years of experience.
The results from the final MCA solution (ten active variables plus two supplementary variables) were subsequently used to perform HCPC. The partitioning of the tree clustering from the HCA presented three clusters (Supplementary File 2: S_Fig 9).
3.4.3. Logistic regression analysis of clinical veterinarians’ biosecurity practices
The univariable logistic regression analysis revealed that veterinarians with a positive perception of the practicality of biosecurity practices had higher odds of washing and sanitizing hands between patients (OR: 2.59; 95 % CI: 1.35–4.95; p-value < 0.001), changing gloves between patients (OR: 2.15; 95 % CI: 1.04–4.48; p-value = 0.04), and isolating animals suspected/diagnosed with zoonotic diseases from other animals and staff (OR: 2.05; 95 % CI: 1.12–3.74; p-value=0.02). The veterinarians who perceived that veterinarians could transmit disease to animals had lower odds of wearing protective clothing (e.g., designated coveralls, shoe cover) while dealing with confirmed cases of zoonotic diseases (OR: 0.42; 95 % CI: 0.20–0.91; p-value=0.02).
4. Discussion
This study evaluated Illinois veterinarians’ knowledge and attitudes toward the risk of foreign and endemic infectious diseases and assessed their disease prevention practices and knowledge of disease reporting by administering an online survey. Our study findings will aid animal and public health authorities in developing effective disease prevention and control strategies.
All responders perceived biosecurity practices and developing biosecurity plans as important for preventing the introduction of an FAD into the US, which is an encouraging finding. They also favorably perceived the importance of disease surveillance and testing for detecting and preventing infectious diseases. However, some veterinarians were not concerned about the likelihood of transmitting an infectious disease from one animal to another when treating sick animals. Similarly, contrasting perceptions were reported by veterinarians when they were asked about the practicality of following biosecurity measures while handling animals in their veterinary clinic or on livestock farms. This is a concerning finding, as veterinarians’ frequent contact with sick animals and noncompliance with effective infection prevention practices such as washing and sanitizing hands between patients might aid disease transmission. Moreover, their perceptions of specific infection prevention practices are important behavior-driving factors, as explained by the theory of planned behavior (Bucini et al., 2019, Heath et al., 2015). Interventions to improve biosecurity practices among veterinarians should also focus on their perceptions to emphasize the impact of their actions on the risk of disease transmission.
The MCA and HCPC analysis revealed that veterinarians in non-clinical professions and veterinarians with biosecurity training were closely linked (clustered) with positive biosecurity perceptions. This finding could be explained by a broader disease risk understanding and perception of biosecurity measures among veterinarians who participated in continuing education courses. In addition, non-clinical veterinarians’ high perception of biosecurity might be explained by their work settings (government or academia), where developing disease prevention and control programs is a priority.
Our results showed that government veterinarians perceived the risk of a FAD outbreak in the US and the possibility of a veterinarian transmitting an infectious disease from one animal to another as higher than veterinarians working in other settings. These differences could be explained by the government veterinarians’ prior experiences and involvement in FAD investigations, disease control, and biosecurity program development.
In the US, veterinarians play a crucial role in disease risk assessment and management (Lipton et al., 2008), and their role requires reporting notifiable diseases and diseases of public health importance to animal and/or public health authorities. Our survey results indicated that most of the veterinarians had inadequate disease-reporting knowledge. However, veterinarians working in non-clinical settings had a higher disease-reporting knowledge than clinical veterinarians. In addition, large animal veterinarians had a higher disease-reporting knowledge than companion animal veterinarians. Similar to our study results, a survey conducted in Arizona, US, in 2015 described that the knowledge of when and how to report a zoonotic disease was lower in companion animal veterinarians than in veterinarians working in other settings (Venkat et al., 2019). The respondents in the Arizona study suggested that it would help them to report diseases if they had easy access to a list of reportable diseases and contact information from reporting agencies. Our study also identified the need to improve and educate Illinois veterinarians about disease reporting. To address this knowledge gap, infographics for reportable swine (Supplementary File 3) and bovine diseases (Supplementary File 4), with information on reporting agencies, were developed and shared in a downloadable format through multiple online platforms, including the ISVMA newsletter and online biosecurity education websites intended for swine (https://vetmed.illinois.edu/swine-biosecurity/resources/
) (Agrawal et al., 2023) and beef (https://vetmed.illinois.edu/beef-cattle-biosecurity/
) producers and veterinarians.
In addition, this study revealed that veterinarians with biosecurity training had a higher disease-reporting knowledge score, indicating that a biosecurity training program for continuing education credits would improve Illinois veterinarians’ knowledge gaps in disease reporting and biosecurity practices.
Differences in the biosecurity practices among clinical veterinary practitioners based on their demographic characteristics were evaluated, and only 73 % of companion animal veterinarians washed and sanitized their hands between patients. This result agrees with the findings of previous studies and emphasizes a need to improve hand-washing practices among veterinarians to increase compliance (Robin et al., 2017, Venkat et al., 2019, Wright et al., 2008). In this study, 55 % of companion animal veterinarians did not change coveralls between sick and healthy patients, which increased the risk of disease transmission. In addition, only 49 % of companion animal veterinarians wore personal protective equipment while treating suspected or confirmed cases of zoonotic diseases (79 % wore only for confirmed cases). This is a concerning finding, which was also described by previous studies indicating veterinarians’ negligence of effective infection prevention practices against zoonotic diseases (Kinnunen et al., 2022; Robin et al., 2017; Williams et al., 2015; Wright et al., 2008). To educate and address this concern, we designed and distributed an infographic on disease prevention and control practices in clinical settings (Supplementary File 5). We chose infographics for extension and education purposes, as these brief graphic illustrations of information attract readers’ attention and enable better retention of information (Ksenija et al., p. 267; Ozdamlı et al., 2016).
We assessed the association between veterinarians’ perception of disease risk and transmission and the practicality of following biosecurity measures in clinical settings. Washing and sanitizing hands, changing gloves between patients, and isolating animals suspected or diagnosed with zoonotic diseases from other animals and staff were found to be more common in veterinarians who perceived biosecurity practices as practical. However, a disconnect between perception and practices was also observed. The veterinarians who perceived that they could transmit disease to their patients had lower odds of wearing protective equipment while dealing with confirmed cases of zoonotic diseases. The plausible explanation for this could come from the tenets of protection motivation theory, where the concern about a potential threat influences the perception of the risk, and coping with the threat influences the course of action (Westcott et al., 2017). The survey respondents might either not perceive the threat to them (but to the animals) or have perceived self-efficacy in coping with a zoonotic disease (Robin et al., 2017, Schemann et al., 2013). Moreover, we did not ask about past experiences with zoonotic diseases, as studies have found that practical experiences are more influential than theoretical knowledge in promoting the use of protective equipment while dealing with a zoonotic disease (Robin et al., 2017).
The survey responders’ demographics corresponded with the veterinarian demographics across the US (AVMA, 2021). In this survey, female veterinarians included 64 % of responders (national average, 65 %), 80 % of responders worked in companion animal medicine (national average, 78 %), and 12 % worked in public and corporate sectors (national average, 15.6 %) (AVMA, 2021).
Before interpreting our study results, a few limitations should be noted. Firstly, the response rate was 8.7 %, which is comparable with other survey studies. Previous studies described that a lower response rate might not affect the validity of the study results (Templeton et al., 1997). The response rate of our survey could be explained by our study design. To increase the study’s reliability, we kept a closed sampling frame by surveying only veterinarians who were members of the ISVMA.
Additionally, a non-response bias (systematic differences between the responses of respondents and non-respondents) should be considered when interpreting the study results. Veterinarians who participated in this study might be more interested in disease prevention; thus, they may systematically differ from those who did not participate. However, the demographics of veterinarians who completed the survey are similar to the characteristics of veterinarians in Illinois and the US (AVMA, 2021), which reduces the non-response bias (Dillman, 1991).
Also, there could be a discrepancy between reporting behaviors and actual practices, particularly with biosecurity and hand hygiene practices (Jenner et al., 2006, Racicot et al., 2012, Robin et al., 2017). An observation-based study that uses video recordings of veterinarians while treating animals would provide more convincing evidence on infection prevention practices implemented in daily clinical practice.
Lastly, we did not ask in detail about the perceived importance of individual practice in infection prevention and control and barriers to the practicality of biosecurity practices. Additional studies could explore these aspects to help guide the development of effective biosecurity methods.
5. Conclusions
Our study described Illinois veterinarians’ attitudes and knowledge toward biosecurity and its importance in disease prevention and control. Knowledge gaps in biosecurity and disease reporting were identified among Illinois veterinarians. Large animal veterinarians and veterinarians working in non-clinical settings had a higher knowledge of disease reporting than companion animal veterinarians. Only half (49 %) of the companion animal veterinarians wore personal protective equipment while treating suspected or confirmed cases of zoonotic infections, despite their concern about the risk of infectious disease transmission. The inconsistencies of veterinarians in abiding by certain infection prevention practices underline the importance of designing directed education programs to improve animal and public health.
Funding
United States Department of Agriculture (USDA) Animal and Plant Health Inspection Service (APHIS) Veterinary Services (VS) – National Animal Disease Preparedness and Response Program (NADPRP) 2020 (Award number: AP21VSSP0000C037).
CRediT authorship contribution statement
Csaba Varga: Writing – review & editing, Supervision, Software, Resources, Project administration, Methodology, Investigation, Funding acquisition, Conceptualization. Isha Agrawal: Writing – review & editing, Writing – original draft, Visualization, Validation, Methodology, Investigation, Formal analysis, Data curation, Conceptualization.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Appendix A. Supplementary material
Supplementary File 1. Supplementary material
Supplementary File 2. Supplementary material
Supplementary File 3. Supplementary material
Supplementary File 4. Supplementary material
Supplementary File 5. Supplementary material
References
- Abdi and Williams, 2010
Principal component analysisWiley Interdiscip. Rev. Comput. Stat., 2 (2010), pp. 433-459, 10.1002/wics.101
View in ScopusGoogle ScholarAgrawal et al., 2023
View PDFView articleView in ScopusGoogle ScholarAlarcon et al., 2021
View in ScopusGoogle ScholarAmerican Veterinary Medical Association, 2021
View in ScopusGoogle ScholarBrown and Bevins, 2018
View in ScopusGoogle ScholarBucini et al., 2019
View in ScopusGoogle ScholarDillman, 1991
Google ScholarDowd et al., 2013
View PDFView articleView in ScopusGoogle ScholarEllwanger et al., 2019
View PDFView articleView in ScopusGoogle Scholar
- Freund et al., 2006
second edRegression analysis, Elsevier (2006)
View PDFView articleGoogle ScholarHeath et al., 2015
- Heiberger and Holland, 2015
Statistical Analysis and Data Display: An Intermediate Course with Examples in {R}(second ed.), Springer-Verlag, New York, New York (2015)
- Husson et al., 2016
Husson, F., Josse, J., Le, S., Mazet, J., Husson, M.F., 2016. Package ‘factominer.’ R Package 96, 698. https://cran.r-project.org/web/packages/FactoMineR.
- Husson et al., 2017
Exploratory Multivariate Analysis by Example Using Rsecond ed. CRC press (2017)
- Jason Bryer, 2016
Jason Bryer, 2016. Analysis and Visualization Likert Items. https://cran.r-project.org/web/packages/likert.
View PDFView articleView in ScopusGoogle Scholar
- Kassambara and Mundt, 2017
Package ‘factoextra.’Extr. Vis. Results Multivar. Data Anal., 76 (2017)(https://cran.r-project.org/web/packages/factoextra)
- Ksenija et al.,
Ksenija, D.-A., Katarina, S., Marijana, T., Marijana, Š., Milan, O., Tijana, S., Monika, S., Dragana, M., Leo, I.,. Community management in serbia and facebook influence on serbian society. https://doi.org/10.1080/19448953.2020.1799594.
View in ScopusGoogle ScholarLovett and Flowerdew, 1989
View in ScopusGoogle ScholarMoore and Lund, 2009
View PDFView articleView in ScopusGoogle ScholarMorley, 2002
View PDFView articleView in ScopusGoogle ScholarNoremark and Sternberg-Lewerin, 2014
- Ozdamlı et al., 2016
Ozdamlı, F., Kocakoyun, S., Sahin, T., Akdag, S., 2016. Statistical Reasoning of Impact of Infographics on Education. Procedia Comput. Sci., 12th International Conference on Application of Fuzzy Systems and Soft Computing, ICAFS 2016, 29-30 August 2016, Vienna, Austria 102, 370–377. https://doi.org/10.1016/j.procs.2016.09.414..
View PDFView articleView in ScopusGoogle ScholarRenault et al., 2018
View in ScopusGoogle ScholarRobin et al., 2017
View PDFView articleView in ScopusGoogle ScholarSahlstrom et al., 2014
View PDFView articleView in ScopusGoogle ScholarSchemann et al., 2013
View PDFView articleView in ScopusGoogle ScholarTrevejo, 2009
View PDFView articleView in ScopusGoogle ScholarVenkat et al., 2019
View PDFView articleView in ScopusGoogle ScholarWeese et al., 2002
Google ScholarWestcott et al., 2017
- Wickham and Wickham, 2017
Wickham, H., Wickham, M.H., 2017. Package tidyverse. Easily Install Load ‘Tidyverse. https://cran.r-project.org/web/packages/tidyverse.
- Wickham, 2016
Ggplot2: Elegant graphics for data analysis (2nd ed), Use R! Springer International Publishing, Cham, Switzerland (2016)
