IPAC 2016Sacramento, CAJuly 31st - August 3rd 2016

Background and margins can also be set for the layout or for the placeholder within the page template.
Use CSS to fine-tune appearance and behavior. <a href="http://help.wildapricot.com/display/DOC/Gadgets#Gadgets-HTMLandCSSparameters/?utm_source=contexthelp&utm_medium=site&utm_campaign=contexthelp" target="_blank" >Learn more</a>

Gold Rush: Nuggets of Truth in Assessment

 

Monday, August 1, 2016

Morning Sessions, 10:30am-12:00pm

Balboa                                                                                                                                       

Technology in selection: When is it appropriate?

Paper Presentation (45 minutes)

Allysa Singer & Megan Krueger

The Personnel Board of Jefferson County

This paper discusses the effects computer-based testing can have on performance on selection exercises. Results of written work sample exercises were compared between hand written responses and responses composed on a computer, leading to the discussion of factors that should be considered when determining the appropriateness of computer-based assessments in selection.

Ongoing monitoring of human capital decisions: Tools you can use.

Paper Presentation (45 minutes)

Martha Hennen, Ph.D.

Consumer Financial Protection Bureau

Presentation describes analytic tools to support ongoing/periodic monitoring of human capital data for fairness and equal opportunity. Discussion covers a compliance monitoring framework developed to engage management in self-evaluation of decisions impacting equal opportunity and diversity. Attendees will gain specific recommendations for methods they can implement in their own organization.


Calavaras                                                                                                                                              

Judging the testing situation: An overview of situational judgment tests

Paper Presentation (45 minutes)

Kelsey Stephens

PAN

This presentation will provide a holistic view of situational judgment tests (SJTs)—including use cases, best practices, and other areas of consideration. The audience will leave the presentation with the knowledge necessary to understand the advantages and disadvantages of SJTs and equipped with the ability to develop and score SJTs.

Situational Judgment Tests: Applications and Advances

Symposium (45 minutes)

John Weiner, Clinton Kelly, & Cyrus Mirza, Ph.D.

PSI

Situational judgment tests have been used as an alternative to traditional paper-and-pencil tests for decades. Numerous studies and meta-analyses have demonstrated that SJTs have useful levels of validity as predictors of job performance. SJTs have been shown to have advantages over more traditional “high-fidelity” practical assessments, such as lower cost and the elimination of rater issues such as training and inter-rater reliability. Due to its popularity, the academic literature and case uses of SJTs in personnel selection are extensive. This symposium will provide a background of SJTs and will review various SJT methods that are available. The symposium will also discuss how technology has helped evolve these assessments and will finish with a discussion of potential directions for future research and application. Within these areas we will discuss specific case studies of SJTs in order to provide real world context and applications.


California                                                                                                                                            

Equal Employment Opportunity in Pay: Regulations, Obligations, and Analysis Strategies

Tutorial (90 minutes)

Michael Aamodt, Ph.D. & Kayo Sady, Ph.D.

DCI Consulting

This tutorial is focused on equal employment opportunity (EEO) regulations and practices that focus on compensation. The presenters review historical and contemporary regulations, commonly implemented analytic strategies, and areas within the compensation EEO landscape in which industrial/organizational psychologists can offer expertise.

 

Diablo

Banding, Adverse Impact and Utility

Paper Presentation (45 minutes)

Frank Igou, Ph.D.

Louisiana Tech University

Test score banding methods define a range of scores in which as scores are considered statistically equivalent. In this presentation, data from a 2015 law enforcement promotional procedure will be examined using standard error of difference bands, based and standard on error of measurement (SEM) and standard error of estimate (SEE). Three different score use rules will be compared: (1) Strict top-down selection, (2) fixed bands, diversity-based selection within bands and (3) sliding bands, diversity-based selection. These methods will be compared for occurrences of adverse impact using the 4/5ths rule and Fisher’s Exact Probability Test. Utility of these methods will also be compared using mean Z scores and the Cronbach-Gleser Utility Formula. Current legal status of banding will also be discussed.

Smith v Boston (2015): The Decision and an Insider's Perspective

Paper Presentation (45 minutes)

Joel Wiesen, Ph.D.

Applied Personnel Research

The decision in this Title VII case takes novel, sometimes reasonable stances on content validation and adverse impact. It both raises and lowers the bar for validation. It broadens the definition of adverse impact. The court rejected a claim of validity generalization for this promotional exam for Police Lieutenant.


El Dorado

The Job Analysis is Complete, Now What?

Tutorial (90 minutes)

Elizabeth Reed

City of Columbus Civil Service Commission

As personnel assessment practitioners we need to apply the science in real world terms. We are charged with devising and implementing meaningful methods of predicting who will be successful in a myriad of jobs. We complete the job analysis and identify the critical knowledge, skills, and abilities for successful performance in the job. We understand the strengths and weakness of various selection tools. But, how do we take this information and determine the best selection process for the job—within the reality of our organization? The selection tools we opt to use depend upon a number of factors. It’s these factors that will drive our decisions. In this session, we’ll briefly discuss strengths and weaknesses various selection tools then focus on the factors that impact our assessment decisions in a real world context. As time permits, we’ll discuss how to guide a change to the context in which we work.


back to top

Monday, August 1, 2016

Student Poster Session, 12:00pm – 1:00pm

Balboa

Revisiting and Refining Interactions: The Effect of a Personality-General Mental Ability Interaction on Job Performance

Alexander Harris & Nathan Carter, Ph.D.

University of Georgia

Anthony Boyce, Ph.D.

Aon Consulting

Michael Tocci

Proctor & Gamble

Despite increasing research on interactionist explanations of job performance, few studies have investigated a personality-general mental ability (GMA) interaction. Support for a personality-GMA interaction has differed and the traits considered have been limited. This study revisits a personality-GMA interaction for predicting performance and considers all Five Factor Model traits.

What do we miss from our nested data? An evaluation of three multilevel models

Li Guan & Nathan Carter, Ph.D.

University of Georgia

Multilevel modeling has gained popularity among organizational researchers. Several measurement approaches are introduced to deal with nested data, but limited research attention is received. To that end, this proposed study compares three multilevel approaches with an aim to suggest an effective approach to better capture nesting and multilevel effects.

Sources of Employer Resistance to Unproctored Internet Testing

Sami Nesnidol & Scott Highhouse, Ph.D.

Bowling Green State University

Assessment practitioners were surveyed regarding reasons for resistance to using unproctored internet testing (UIT) for pre-employment tests. Concerns about data quality and diffusion in the field were significant predictors of resistance. Public sector practitioners were found to be significantly less likely to choose UIT than private sector practitioners.


back to top

Monday, August 1, 2016

Afternoon Sessions, 1:30pm – 3:00pm

Calavaras                                                                                                                                              

Fits and Starts: The Evolution of Testing for the State of California

Invited Session (60 minutes)

Adria Jenkins-Jones, Chief, Selection Division & Bryan Baldwin, Chief, Personnel Management Division

Organization: California Department of Human Resources

In this lively discussion, the presenters will discuss the current state of employment testing for the state of California, including significant recent and upcoming changes to our examination software. Using an interview format, the presenters will discuss the massive changes envisioned for statewide testing, and how the California Department of Human Resources is attempting to collaborate with stakeholders to change the traditional paradigm, and significantly improve automation, while maintaining the commitment to merit. The presenters will engage in open dialogue and there will be time for audience members to ask questions about current and future directions of testing for the state.


California                                                                                                                                            

Contemporary Content-Oriented Validation Research: Same Old Song and Dance?

Panel (90 minutes)

Michael Aamodt, Ph.D. & Emilee B. Tison, Ph.D.

DCI Consulting Group

Kathryn Niles-Jolly, Ph.D.

U.S. Office of Personnel Management

Heather Patchell, M.A.

Biddle Consulting Group

Scott C. Whiteford, Ph.D.

Talent Plus, Inc

Organizations often employ selection tools to make personnel decisions. The value of these tools depends on whether or not accurate inferences about applicant or employee characteristics can be made based on the selection procedure scores. Content-oriented validation research is a commonly used and potentially useful organizational strategy to support the use of selection procedures, both from organizational value and defensibility perspectives. However, I/O Psychologists have recently debated how useful content-oriented strategies are, particularly relative to other validation strategies. Additionally, recent court cases (e.g., Lopez v City of Lawrence, 2014) have put content-oriented research in the spotlight. Although content-oriented validation strategies are recognized as legitimate and legally defensible methods for test development and evaluation, this panel will provide an updated discussion of content validation in today’s organizational and legal environments. Particular emphasis will be on modern challenges, practical constraints, defensibility considerations, and value of this approach relative to other strategies.

 

Diablo

Transitioning to computer-based testing: Lessons learned and recommendations

Paper Presentation (30 minutes)

Emily Nguyen & Tomeca Turner

U.S. Secret Service

This paper presentation will discuss the U.S. Secret Service’s migration from wholly paper and pencil testing to computer-based testing for its entry-level selection process. This presentation will address expected and unforeseen challenges to implementing computer-based testing as well as practitioner-focused strategies and recommendations for data management and safeguarding applicant PII.

A Moving Target: Test Security in Temporary Facilities

Paper Presentation (30 minutes)

Brian Ross & Asia Johnson

U.S. Secret Service

This presentation will provide views on how the U.S. Secret Service ensures the security of entry level and promotional exams both domestically and internationally. It will also discuss the potential repercussions of inadequate test security protocols and unsecure testing environments.


El Dorado

Job Analysis: Methods & Implementation

Tutorial (30 minutes)

Tomer Gotlib, Ph.D. & Julie Sampson, Ph.D.

Ergometrics & Applied Personnel Research, Inc.

This tutorial will cover methods and best practices for job analysis with the intent of educating attendees on how to effectively and efficiently collect information about a job. The tutorial will also give attendees a chance to practice writing test questions with the knowledge gained during the tutorial.

Developing Physical Ability Work Simulations

Tutorial (30 minutes)

Oscar Spurlin, Ph.D.

Ergometrics & Applied Personnel Research, Inc.

In our many years of work in design of physical ability tests for industrial and public safety occupations, our methodology has come to stress work simulation tests over standard exercise test formats. This paper will present reasons for this preference, describe the steps in test design, and discuss some of the pitfalls in trying to create effective simulations.


back to top

Tuesday, August 2, 2016

Morning Sessions, 10:30am – 12:00pm


Calavaras                                                                                                                                              

The Challenges of Measuring Interpersonal Communication Skills

Symposium (45 minutes)

James Kuthy, Ph.D. & Heather Patchell, M.A.

Biddle Consulting Group

Interpersonal communication skills are frequently found to be an important factor for success for many jobs. However, there appear to be potential limitations to the validity and reliability of some of the frequently used tests assessing these skills. The presenters will discuss real-world issues they’ve encountered implementing valid and reliable measures of interpersonal communication.

Using Exit Survey Data in Organizational Health Assessment

Paper Presentation (45 minutes)

Martha Hennen, Ph.D. & Tanisha Robinson

Consumer Financial Protection Bureau

This presentation will describe compiling, reporting and using Exit Survey data across time to inform assessment of organizational health. Comparison of exiting employee perceptions by demographic and functional variables is used to inform human capital business partnership, risk assessment and continuous improvement.


California                                                                                                                                            

Perspectives on Orlando's Mass Shooting: What can I/O Psychology do?

Invited Session (90 minutes)

Deonda Scott

City of Orlando

The City of Orlando was recently rocked to its core by the nation’s deadliest mass shooting to date. The horror that unfolded is strikingly similar to our most recent police promotional emergency incident command exercises. This session will explore how the City of Orlando and other organizations have used interactive emergency incident command simulations to select leaders who will be best able to save lives during times of crisis.

 

Diablo

This job analysis is tasking: A different approach to conducting job analysis to support physical fitness and medical standards

Paper Presentation (45 minutes)

Kathlea Vaughn & Kathy Stewart

U.S. Customs and Border Protection

CBP was faced with the challenge of conducting job analyses of three law enforcement occupations to support the development of medical and physical fitness standards with the goal that the physical standards be equivalent. The presenters will describe the job analysis model developed and specific challenges faced and solutions implemented.

Teaching an Old Dog New Tricks: Moving from Face-to-Face Job Analysis to a Distance Approach

Paper Presentation (45 minutes)

Brooke Parker, M.S., Dennis Pribe, M.A. & Bob Mahlamn, M.A. PMP

The Ohio State University

Our legacy job analysis approach has been highly effective, but not without drawbacks for geographically dispersed organizations and those who cannot afford staff to be offline for extended periods of time. Presenters will compare and contrast the legacy and distance approaches, sharing challenges and design considerations encountered along the way.


El Dorado

Strengthening the HR foundation: Creating career tracks for your HR Workforce

Paper Presentation (45 minutes)

George Johnston

U.S. Secret Service

The U.S. Secret Service sought to improve the career development opportunities for its Administrative, Professional, and Technical (APT) workforce through its APT Career Track Initiative. A thorough look at the creation of the HR Specialist and Instructional Systems Specialist pilot programs will provide insight on the lessons learned.

back to top

Tuesday, August 2, 2016

Afternoon Sessions, 1:30pm – 3:00pm

Calavaras                                                                                                                                              

The Trainer Matters: Cross-Classified Models of Trainee Reactions.

Student Award (45 minutes)

David Glerum

University of Central Florida

A vast number of human resources professionals have been utilizing trainee reactions (Kirkpatrick, 1959) as part of their training evaluation systems (American Society for Training & Development, 2009). Although prior research (Alliger, Tannenbaum, Bennett, Traver, & Shotland, 1997; Brown, 2005) has examined the factor structure of trainee reactions, little is known about whether or not variability in trainee reactions reflects differences in trainers (i.e., who conducts the training) or training content (i.e., what is being trained). Using cross-classified random effects models (CCREM; Rasbash & Goldstein, 1994; Raudenbush, 1993) of trainee reactions from over 10,000 employees enrolled in professional development programs, we found that variability across trainers is the primary source of variation in trainee reactions (in comparison to variability in training content), suggesting the trainer influences reactions more than what is being trained. Based on our results, we provide practical and research implications as well as directions for future research.

Capital One & Quintela Group eInterview System

Innovation Award (45 minutes) 

Julia Bayless 

Capital One 

Adam Vassar

Quintela Group

In response to the clear need for behavioral interviewing to enter the HR technology age, Capital One and Quintela Group embarked upon an online interview system project to digitize the Capital One interview program. The resulting eInterview system represents a significant technology innovation for interviewing practices. The system includes: a Builder & Guides Portal and a Candidates & eGuide Portal. This presentation will discuss the system development and implementation, lessons learned and results achieved.


California                                                                                                                                            

What if and what now: A critical look at the use of content validation methods

Panel (90 minutes)

Erin Wood

PAN

Elizabeth Reed

City of Columbus Civil Service Commission

Kathlea Vaughn

U.S. Customs and Border Protection

Heather Patchell, M.A.

Biddle Consulting Group

Kelsey Stephens

PAN

The ability to validate the use of assessment tools is paramount for the ability to create fair and defensible employment programs, and one method frequently used to validate tools used in assessment programs is content validation. Although content validation has been supported as appropriate for use in validating employment measures by both the SIOP Principles and the Uniform Guidelines, in the field of Human Resources and Industrial and Organizational Psychology, content validation standards have not become as uniform as those related to criterion-related validation. As such, the goal of this panel will be to examine content validation as a method through a critical lens, identifying weaknesses and concerns associated with the method, and how experienced practitioners address those issues. The panel intends to create an open discussion with the audience about the realities and limitations associated with conducting content validation studies, and discuss methods for addressing difficult questions and decisions.

 

Diablo

Improving Candidate Experience: Robust Tactics for Implementing Video Interviewing

Tutorial (90 minutes)

Daniel Schmeling, Ph.D., Anne Scaduto, Ph.D., Jen Harvel, Ph.D. & Maureen Costello, M.A.

Capital One

Technology has been leveraged in the hiring process recently by using one-way video interviews where candidates record themselves responding to pre-established sets of questions that are viewed by evaluators at a later time. Research has shown the effectiveness of this technology by comparing one-way video interviewing to traditional phone and in-person interviewing. One-way video interviewing has been shown to be a superior option in terms of the candidate experience, hiring manager experience, efficiencies in the hiring process, cost savings, and quality hiring decisions (Scaduto, Schmerling, & Harvel, 2016). This tutorial will provide a step-by-step guided set of examples and instructions on how to use one-way video interviewing best practices to implement this technology in an organization. The authors will provide role-play exercises and materials that explain and show to session attendees how one-way video interviewing can be efficiently and effectively introduced into any organization large or small.


El Dorado

Affective Assessments - The Missing Link in Employee Engagement

Tutorial (90 minutes)

Akeela Davis

Courageous Business Culture

Cognitive assessments show what people can do. Conative assessments show how their brains work in challenging situations. Affective assessments, the missing link, are feelings based, and show what drives people to do, or not do. So how can adding an affective assessment, enhance Employee Engagement in organizations? This session discusses three ways it impacts on communications. First, it makes available the true, current, and changing, voice of employees to the organization. Second, because of that voice, it offers managers a conjoint basis to develop employees. Third, because engagement is feelings based, it shows how opposing motivations create tensions between individuals and within teams. We will discuss how these can be diffused, and the results measured in concrete metrics.

back to top


Wednesday, August 3, 2016

Morning Sessions, 8:30am – 10:00am

Calavaras                                                                                                                                              

Validaiton of Pre-Employment Screenings

Tutorial (60 minutes)

Kathleen Walker

City of Little Rock

Program Information: This presentation will cover basic content validation techniques to ensure that all pre-employment screenings are both content valid and EEOC compliant. Recent and applicable court cases will be discussed to ensure that best practices are utilized, including the application of the Green Factors for background checks. Finally, participants will be provided a step by step validation process with a sample of the required documentation that they can utilize in their agencies.

Visibility in Interviewing: Video Outperforms Phone Interviewing Outcomes

Paper Presentation (30 minutes) 

Anne Scaduto, Ph.D., Daniel Schmerling, Ph.D., Jen Harvel, Ph.D. & Maureen Costello, M.A.

Capital One

Abstract One-way video interviews are being leveraged in the hiring process where candidates record responses to a set of questions viewed by evaluators. This study explores the effectiveness of this technology by comparing one-way video interviewing to traditional interviewing within a large, Fortune 500 Company in the financial services industry.


California                                                                                                                                            

Assessing and Developing Executives’ Business Acumen

Symposium (90 minutes)

Ted Prince, Ph.D.

The Perth Leadership Institute

Traditional competency assessments do not specifically focus on innate financial competencies and business acumen. This presentation discusses these and the nine financial signatures of executives. It shows how each of these impacts financial performance and valuation outcomes of organizations. Based on Dr. Prince’s book “The Three Financial Styles of Very Successful Leaders” (McGraw Hill, 2005) the presentation discusses how to identify and measure financial impact and business acumen using formal assessment instruments and how to use these findings to improve company financial performance and valuation.

 

Diablo

State of Federal Hiring: Update & Future Directions

Symposium (90 minutes)

Michelle Earley, M.B.A., Margaret Barton, Ph.D.,

Patrick Sharpe, Ph.D., & Jennifer Demarais, M.S.

U.S. Office of Personnel Management

Ryan O’Leary, Ph.D.

PDRI

John Ford, Ph.D.

U.S. Merit Systems Protection Board

Some contend that the process for hiring federal employees is broken – that the government is unable to hire and keep top talent. Given the volume of applicants for federal positions and the high-stakes nature of many mission-critical positions, there is an ongoing need to maximize effectiveness and efficiency, while satisfying a wide range of stakeholders. The purpose of this symposium is to report on what is working and what is not working in federal hiring. Presenters representing government, private sector, systems, research, and practice perspectives will share key updates, challenges and opportunities surrounding this critical hiring sector.


El Dorado

Best Practices for Generating Test Specifications from Job Analysis Results

Symposium (60 minutes)

Jason Schaefer & Clinton Kelly

PSI Services

Howard Forston, Ph.D.

East Bay MUD

The results of job analyses provide the foundation for the development of valid and job related examinations. An important step in this process is the development of test specifications that provide a linkage between the job analysis results and the exam’s content and weighting (i.e., the quantity of items or points assigned to each examination topic area). The presenters will provide a brief overview of the literature for using job analysis data to develop test specifications and their implications for the design and execution of job analysis studies. Additionally, a comparison of the various methods available for mechanically combining job analysis and linkage rating results to generate preliminary KSA weights will be explored. Finally, the presenters will discuss how these preliminary weights can be used to guide and facilitate the committee’s specification development discussions ultimately providing a stronger empirical linkage between the job analysis results and the finalized exam specifications.

back to top

Powered by Wild Apricot Membership Software