• Overview
• Survey Pre-Test
• Survey Response Rate
• Data Recording/Transfer
• Earl Survey Research Lab
• Key Personnel Bios

The Regional Workforce Survey (RWS hereafter) is an establishment survey that collected information about skills, training, qualifications, staffing patterns, recruitment, salary and benefits for occupations in the 29 county workforce commuting region of Lubbock, TX. We identified establishments within this region with at least 5 employees using standard commercial resources. A stratified sample was constructed to ensure a representative sample across three geographic regions and industrial sectors defined by 2-digit NAICS codes. Establishments were randomly selected from a representative sample of establishments within this sample frame. Interviewers in the Earl Survey Research Laboratory at Texas Tech University’s Department of Political Science conducted survey interviews that lasted about 12 minutes on average. The response rate for the survey was 51.5%, which means that 3,130 establishments participated in the survey. After the data were cleaned and entry errors removed, the survey contained responses from 3008 establishments about 520 occupations within the region. There are responses from at least five different establishments for 270 occupations.

This survey uses a stratified sampling process to ensure a representative sample of establishments across the workforce commuting region. Industrial classification by two-digit NAICS codes is the first stratum. The NAICS industry classification provides a more accurate description of modern economies, and all government agencies are completing the transition to these codes for economic and labor data.

The second stratum is based upon geography. The geographic stratification creates three regional zones that are used to ensure a proportional sample (see Table 1 and Figure 1). Zone 1 is Lubbock County. Zone 2 includes all counties within the South Plains Workforce Development Area (SPWDA) except Lubbock County. Zone 3 includes the 14 counties peripheral to the SPWDA. The data can then be presented in a layered fashion so end-users can focus upon Lubbock County, the SPWDA (including Lubbock County), or the entire workforce commuting region (Zone 1 + Zone 2 + Zone 3).

This stratified sampling produced a representative sample of establishments within each zone and within the entire workforce commuting region. Table 2 provides a comparison of the actual distribution of establishments based on data from an industry standard marketing resource and the sample distribution of establishments by two-digit NAICS and regional zones. Each row indicates an industry by a two-digit NAICS code. Each cell within the row indicates the distribution of establishments across geographic zones. The table reports both the distribution of establishments identified with standard marketing resources and the distribution of survey respondents. A comparison suggests the sample is representative of the region’s establishments by industry and geographic location. The RWS does not stratify on business size. Stratification by industry type and geography captures a representative sample of business sizes as demonstrated in Table 3.

In sum, the methodology of the RWS provides high quality, timely data from a local source. Questions regarding details of the survey methodology and implementation should be directed to project investigators, Aman Khan or Brian K. Collins of Texas Tech University. A report of key findings is available on the summary of the survey page.

Go back to top

Pretesting for the instrument took place on December 29-30, 2003 and January 13-14, 2004. Because the questions comprising the instrument were asking for basic, standard business information, the main purpose of the pretesting was to determine an average length for the survey interview, accounting for various sizes of businesses. Therefore, business size was the critical factor in selecting businesses to complete a pretest interview. There were no region-specific questions, so regional representation in the pretest was not considered essential. In fact, the businesses used for the pretest were outside of the study area entirely in order to avoid using “live” sample for the pretest. This was of particular concern among large businesses, as there were but a handful of such businesses in the Lubbock workforce commuting region.

A total of 8 pretest interviews were conducted with at least one business from each of three size categories (up to 29 employees, 100-250 employees, and 1,000-5,000 employees). These ranges were chosen to compare small, medium and large businesses. We expected to have varying average interview lengths because the length of the interview was dependent on the number of occupations that were employed by a business. The pretest results indicated that the average interview length was on target and that there were no questions that respondents had difficulty interpreting or answering.

The second portion of the pretest consisted of a thorough testing of the instrument to ensure that there were not programming errors that would, for example, result in incorrect branching patterns. Each question on the instrument was answered multiple times using different responses to reveal any programming errors as just described, as well as display problems and issues related to customization of the questionnaire (e.g., making sure the correct occupational title appeared as intended).

As mentioned previously, there were no concerns about regional representation in the pretest because all of the questions were specific to the individual business and not dependent on geographical location. The pretest subjects were located beyond the study’s catchment area.

Nothing in the pretest results suggested a need for modification of interviewing protocol or substantive items in the questionnaire. The instrument that was pretested was the instrument employed in the actual study.

Go back to top

Interviewing commenced on Thursday, January 22 and continued through Friday, March 26 for a total data collection period of nine weeks. The average length of interview was 11.3 minutes.

A completed interview was obtained from 3,130 businesses. This represents 43.0% of the sample population of 7,275. The entire sample population was actually 9,354 businesses, but the desired number of completed interviews was obtained quickly enough that 2,079 businesses were not attempted. In other words, because the cooperation rate exceeded expectations, the entire available sample was not needed to secure the target number of interviews. The cooperation rate (completed interviews /[completed interviews + refusals]) was 81.8%. Put another way, for every refusal received, about 4.5 interviews were completed. The response rate (completed interviews / total valid sample size) was 51.5%. In other words, an interview was obtained from 51.5% of the valid sample, and the rest were cases in which a respondent could not be reached to complete an interview, whether due to scheduling conflicts or simply a general lack of availability.

Respondents from Lubbock County make up 50.5% of the respondents, and 46.3% of the civilian labor force in the study area are located in Lubbock County. Respondents from the South Plains Workforce Region make up 18.1% of the respondents, and 18.2% of the civilian labor force in the study area are located in this region. Respondents from the area within the workforce commuting region of Lubbock, but outside the South Plains Workforce Region, make up 31.4% of the respondents, and 35.5% of the civilian labor force in the study area are located in this peripheral region. Therefore, it appears that the final data set is quite representative of the civilian labor force for the study area. Because the response and cooperation rates are high for the study overall, there is no reason to suspect significant differences in response rates across regions.

The data were also examined for representativeness at the industry-sector level and it was found that the data are adequately representative at this level. Therefore, no post-collection adjustments to the data were made.

Go back to top

The data were collected utilizing a computer assisted telephone interviewing (CATI) system. An interviewer, seated at a computer workstation, reads the questions that appear on the screen to the respondent. Each response category is numbered on the screen, so the interviewer simply enters the number that corresponds to the answer provided by the respondent. For open-ended questions, the interviewer types the answer into a text box that appears on the screen. The questionnaire is programmed for the CATI software so that appropriate skips and branches are carried out, and appropriate customized displays appear when relevant. The data are automatically saved to the survey laboratory’s file server, meaning all respondent data are saved to the same centralized file. The entire file server is backed up nightly to a remote and secure location by the university’s office of Technology Operations and Systems Management. The raw file itself is only readable by the CATI software, but can be exported to a number of different file types such as Excel, ASCII, SPSS, etc.

The CATI software processes the data to make sure the file’s integrity is intact, and exports the data to a format of the user’s choice. In the present study, the data were exported to SPSS for Windows for further processing and analysis. Microsoft Excel was also used to merge business-specific information (e.g., county, region) into the main data file.

The CATI software consists of two distinct programs that work in conjunction with one another. The first is Ci3 which is the interviewing software. This is the program that is used to enable the questionnaire to appear on the screens of the interviewers’ workstations and is the program that actually collects and saves the data. This program also exports the data, transforming it into a file readable by other programs such as Excel and SPSS. The second program, called WinCATI Supervisor, manages the sample when the study is in the field. For example, if an interviewer calls a business and the respondent asks to be called back the next day at a certain time, the interviewer enters that date and time at their workstation and WinCATI Supervisor ensures that the record is delivered to a workstation on the specified date and time. It also monitors interviewer productivity, records the number of calls placed by each interviewer (and number of interviews completed), and controls access to the survey projects so that interviewers who are not qualified to work on a particular study are not able to work on it. Both Ci3 and WinCATI Supervisor are produced and distributed by Sawtooth Technology of Northbook, IL. Sawtooth products are among the most widely used in the survey research industry.

Over the course of the study’s data collection period, forty-two interviewers worked on the project. The interviewers were supervised by at least one of a team of four supervisors at all times. This supervision included both video and audio monitoring. Supervisors had the ability to view what appeared on all interviewers’ screens and could listen to the interview unobtrusively from a remote location. This ensured that the interviewers followed study protocol and accurately entered responses.

Most of the interviewers for the project were Texas Tech University students, although some were members from the community with no other university affiliation. The formal training for interviewers on this project consisted of two parts. The first was general interviewer training, which all interviewers complete. All interviewers are thoroughly trained in the proper administration of telephone surveys, including gaining consent, respondent selection, reading questions and recording responses, case disposition, refusal avoidance, refusal conversion, professionalism, and research ethics. The second part of the training consisted of project-specific training, where interviewers were informed about the background and goals of the project, and every question on the survey instrument was reviewed and discussed so that interviewers would be thoroughly familiar with the questionnaire and would be in a position to make appropriate decisions when necessary during an interview. The interviewers were also provided with a study-specific guide for the study that gave information on the purpose and sponsors of the study, instructions for particular survey items and definitions of important terms. Following the formal training, interviewers engaged in self-directed training to become familiar with the CATI software and become accustomed to properly asking questions and recording responses. All interviewers were required to pass a written quiz covering material from the formal training and successfully complete a challenging mock interview with a supervisor before being cleared to call actual respondents. Many interviewers were denied clearance after their first mock interview and were required to continue with self-directed training until a supervisor indicated they could make a second attempt at the mock interview. Interviewers who could not satisfactorily complete the mock interview after two attempts were terminated from employment (this was the case with only a small number of interviewers).

Go back to top

Established with a gift from Lewis and Maxine Earl of Post, Texas, the Earl Survey Research Laboratory (ESRL) is housed in the Department of Political Science at Texas Tech University. The lab has conducted survey research projects for internal clients (Texas Tech University Office of the President; Student Union; Diversity Strategic Planning Committee; individual faculty), government entities (Lubbock City-County Libraries; Texas Office of Rural Community Affairs; Texas Department of Transportation [forthcoming]; City of Lubbock Parks & Recreation Department; City of Amarillo Parks & Recreation Department), and other universities (University of Nebraska-Lincoln; Georgia State University). The Laboratory maintains the highest standards for data quality, research methodology, and research ethics.

The Laboratory is equipped with 21 interviewing stations and a supervisor station. These workstations feature state of the art computers running the widely used Ci3 and WinCATI interviewing and CATI software from Sawtooth Technologies. Data are stored on the Lab’s secure file server, which is fully backed up 7 nights a week. Interviewers work four-hour shifts in which at least one supervisor is present with additional supervisors as necessary. Video and audio monitoring occurs continuously throughout each shift to ensure proper interviewing technique and data quality. The ESRL employs staff interviewers who are fluent in Spanish for studies that require that the questionnaire be available in Spanish for respondents who prefer to respond in that language.

Go back to top

Aman Khan, Ph.D., is Associate Professor of Political Science and Public Administration at Texas Tech University, where he teaches public budgeting, financial management, and quantitative methods. Trained as an economist and planner, he has an M.A. in Economics, an M.S. in Urban and Regional Planning, and a Ph.D. in Public Administration. He has previously served as Director of the Graduate Program in Public Administration at Texas Tech and currently serves as Research Associate for the Center for Public Service as well as on the editorial board of several public administration journals. Dr. Khan has authored several books and contributed works to various edited collections and professional journals.

Brian K. Collins, Ph.D., is Assistant Professor of Political Science, Research Associate for the Center for Public Service, and Director of the Graduate Program in Public Administration at Texas Tech University. He has published research regarding corporate tax policy, health care policy and public finance. He was also the principal investigator for a research project regarding the cost-effectiveness of relocating the Texas Office of Rural Community Affairs. Dr. Collins has conducted state-wide surveys of rural Texas residents, political and economic leaders in rural Texas, and a 50-state survey of state immunization program directors. Other research interests include rural community development, rural health care policy, and the impact of political institutions on economic development in rural regions.

Brian Cannon, director of the Earl Research Laboratory at Texas Tech, holds a master’s degree in Sociology from Penn State University. He has served as a project manager for the Bureau of Sociological Research at the University of Nebraska-Lincoln, and an Assistant Director of the Survey Research Laboratory at Georgia State University’s Applied Research Center. Mr. Cannon has overseen survey research projects utilizing various data collection methodologies, including telephone, mail, internet, face-to-face, and focus group interviewing. In addition, he has managed projects varying in size and complexity, including a two-year study sponsored by the National Institute of Mental health that incorporated several methodologies and survey designs. Mr. Cannon also served as co-chair for the International Field Directors and Technologies Conference and is a member of the American Association for Public Opinion Research.

Doug Nutsch is the Lead Programmer Analyst of the Teaching and Learning & Technology Center at Texas Tech University, where he assists in developing distance learning technology, teaches faculty focused short courses on Macromedia Dreamweaver, and consults on web technologies such as XHTML, classic ASP, ASP.Net, XML, and various backend applications. He holds a master's of science degree in Management Information Systems from Texas Tech University and has worked on numerous web centric technologies throughout his career with Accenture and Texas Tech.

Go back to top