Reliability and Validity Question

Instructions, rubric, and Values and Motives Questionnaire (VMQ) attached

COUC 521

Benchmark Benchmark Exploring Reliability and Validity Assignment Instructions

Overview

In this
Benchmark Exploring Reliability and Validity Assignment, you will analyze the psychometrics provided from the Values and Motives Questionnaire (VMQ), addressing 1) the type of reliability and validity used; 2) the areas of concern/strength for the Cronbach alpha coefficients; 3) how sample size and nature of the population may influence the constructs that the test attempts to measure (including validty and norming population); and 4) present an opinion of the test using the information in this report.

Instructions

·
Length of
Benchmark Exploring Reliability and Validity Assignment
: 600-900 words (not including the title page and reference page)

·
Format of
Benchmark Exploring Reliability and Validity Assignment
: APA for font (Times New Roman, 12 pt.), title page, margins, and section headings

·
Number of citations: 2+ (must include the VMQ technical manual)

·
Acceptable sources: scholarly articles/texts published within the last five years

Template

1.
Types of Reliability and Validity.

Explain the types of reliability and validity used. The authors of the VMQ explain general principles of norming, two types of reliability, and two types of validity on pages 14-15. But when they present the specific information about reliability in the first paragraph on page 17, they specifically state that they are using one type of reliability and where this data can be found. This is the reliability that you will describe. The same applies to validity; find the specific information provided by the authors on the VMQ.

2.
Reliability: Cronbach Alpha Coefficients.

Look at each of the constructs (scales) the test purports to measure and identify the proper category of low, acceptable, and high ranges of the
Chronbach’s alpha coefficients for
each scale. Use the chart provided in the “What Makes a Good Test” handout, which explains how to read reliability and validity coefficients. Match them to what is in the manual. Report the reliability category for all scales. Look at the number/percent of the scales with problematic versus acceptable internal consistency
alpha values.

3.
Sample Size and Nature of the Population.

a.
Validity: Look at the population used for the VMQ and the populations for the tests used to evaluate the VMQ’s validity. Do you believe that the populations of the other tests are comparable to the population used in the VMQ? Does this raise an issue with the validity?

b.
VMQ Norming Population: Is it representative of the population you want to use it for? How about gender and cultural representation? Is there demographic information and is it sufficient to draw conclusions? Do you believe that the results of the group taking the VMQ are broad enough to generalize to other populations? Did they sample enough people to believe that this sample captures the traits of the population? The answers to these questions will help you to evaluate potential issues with both validity and generalizability.

4.
Your Opinion of the VMQ.

Summarize the psychometric properties you reported. Look at your evaluation of reliability, validity, and population. Based on your findings, do you believe that the VMQ’s authors sufficiently establish the reliability and validity of the VMQ? Note any concerns. Be sure to view this in terms of the sample size and the makeup of the populations used to norm the instruments. This adds important meaning to the reliability and validity coefficients.

To access and download the
VMQ Technical Manual, see the
Benchmark Exploring Reliability and Validity Resources.

Be sure to review the criteria on the
Benchmark Exploring Reliability and Validity Resources Grading Rubric before beginning this
Benchmark Exploring Reliability and Validity Resources Assignment.

Note: Your assignment will be checked for originality via the Turnitin plagiarism tool.

Page 2 of 2

the
te

c

hnical
manual

5

VALUES and
MOTIVES
QUESTIONNAIRE

Measures of Personality Values & Interests

ONTENTS
1 THEORETICAL OVERVIEW

2

VALUES SCALES CONTAINED WITHIN THE

VMI

3 PSYCHOMETRIC PROPERTIES OF THE VMI

4

ADMINISTRATION INSTRUCTIONS

5 REFERENCES

c

2

3

1 MEAN RAW SCORE DIFFERENCES & T-TEST SIGNIFICANCE LEVELS BY GENDER

2 VMI INTERNAL CONSISTENCIES & ITEM TOTAL CORRELATIONS (ITC’S)

3 VMI INTER-CORRELATION MATRIX

4 CORRELATIONS BETWEEN VMI AND MAPP

5 CORRELATIONS BETWEEN VMI AND 16PF FORM 5

6 CORRELATIONS BETWEEN VMI AND OPP

LIST OF TABLES

4

1THEORETICAL
OVERVIEW

Values are presumed to encapsulate
the aspirations of both individuals

and societies. They relate to the most
desirable, deeply ingrained stan-

dards that determine future direc-
tions and explain past actions.

Values have been treated as key con-
structs in the process of socialisation,
and have emerged in research in the

occupational, cultural, religious,
political, educational areas. Other

intellectual traditions view values as
also having an individual function
shaped by the biological and psy-

chological needs of each person. This
perspective has fostered research

linking values to the attitudes and
personality of individuals and to the

maintenance and enhancement of
self-esteem. In spite of widespread

acceptance of the relevance of values
to human activity at both the indi-
vidual and social levels of analysis,
developments in the field have been
hampered by problems of definition
and doubts about the empirical via-

bility of the construct.

1 THE CONCEPT OF VALUES AND ITS

ROLE IN PERSONNEL ASSESSMENT

6

THE CONCEPT OF VALUES
AND ITS ROLE IN
PERSONNEL ASSESSMENT
Out of numerous deliberations in the
l950s and 1960s a unifying consen-
sus emerged that values were
“person-centered” and pertained to
the desirable.” a consensus captured
in the following definition:

A value is a conception, explicit or implicit, distinctive of an individual or
characteristic of a group, of the desirable which influences the selection from

available modes. means, and ends of action.

In spite of a unifying theme at the
conceptual level, convergence in
empirical values research did not
follow. One of the concerns was
appropriate level of abstraction for
sampling value items. Values were
widely accepted as general rather
than specific. It has never been clear,
however, whether values were to be
inferred from responses to specific
attitude statements or more directly
from general orienting responses.
Furthermore, at what point on the
specific-general continuum did atti-
tudes become values?

It was only during the early
1970’s that a conceptual and opera-
tional framework that had eluded
value research was proposed.
Rokeach (1973) defined a Values as
“… an enduring belief that a specific
mode of conduct or end-state of exis-
tence is personally of socially
preferable to an opposite or converse
mode of conduct or end-state of exis-
tence”. Sets of values formed Value

systems, defined as “enduring organ-
isations of beliefs concerning
preferable modes of conduct or end-
states of existence along a continuum
of importance”

These value systems were regarded
as part of a functionally integrated
cognitive system’ in ‘which the basic
units of analysis are beliefs. Clusters
of beliefs font attitudes that are func-
tionally and cognitively connected to
the value systems. Rokeach further
postulated classes of beliefs concerned
with self-cognitions representing “the
innermost core of the total belief
system and all remaining beliefs, atti-
tudes and values can be conceived of
as functionally organised around this
innermost core. Like other beliefs,
then, values serve to maintain and
enhance the self-concept.

7

Rokeach accepted values as
general beliefs, as having a motiva-
tional function, as not merely
evaluative but prescriptive and
proscriptive, as guiding actions and
attitudes, and as individual as well
as social phenomena. As well as
consolidating these themes, Rokeach
integrated a number of other strands
of thought and research from the
values literature. A significant body
of work has focused on the attitude-
value relationship. According to this
view, values were more central
concepts than attitude. were deter-
minants of attitude, and were more
resistant to change, with favourable
attitude emerging toward objects
instrumental in the attainment of
important values.

8

2VALUES SCALES
CONTAINED WITHIN
THE VMI

To ensure a comprehensive coverage
of the universe of occupationally rele-

vant value items, a review was
undertaken of research and instru-
ments in the Values arena. On the

basis of this review, three Value cate-
gories were formulated:

Interpersonal
Extrinsic
Intrinsic

1 INTERPERSONAL VALUES

2 EXTRINSIC VALUES

3

INTRINSIC VALUES

bk INTERPERSONAL VALUES
Interpersonal

Values that refer to
relations with others.

Extrinsic
Values that refer to

motivating factors at
work.

Intrinsic
Values that relate to
personal beliefs and

attitudes.

ALTRUISM

High scorers on altruism tend to be
generous and helpful people, always
ready to do something for other
people. They will be inclined to be
easily moved by the plight of those
less fortunate than themselves and
will, if able, attempt to do something
to help alleviate the suffering the see
around them.

Low scorers on altruism will be
inclined to have a less sympathetic
attitude toward the plight of those
less fortunate than themselves.
Believing, perhaps, that most people
are responsible for the position they
find themselves in low scorers see
little reason why they should help
those in a less fortunate position.

AFFILIATION

Those people scoring highly on need
for affiliation require a great deal of
contact with other people. High
scorers will go out of their way to
meet people, feeling at their most
comfortable when engaged in activi-
ties involving other people. With a
need for the companionship of other
people, friends and associates play a
very important role in their life.

Low scorers tend to be much
more self-sufficient with little need
for the companionship of others.
Tending to prefer solitary pursuits
they will be comfortable with their
own company.

AFFECTION

High scorers have a tendency to
want to get close to people, liking
others to show warmth and affec-
tion. With a need to be able to share
feelings and emotions with sympa-
thetic others high scorers will tend to
fairly empathic.

Low scorers have no particular
wish to get too close to people.
Having no particular desire for
others to show sympathy or concern
towards them they will be unlikely to
display these sentiments towards
others.

blEXTRINSIC VALUES
ACHIEVEMENT

High scorers on achievement tend to
want to excel in everything they
attempt, no matter what this might
cost. With a desire to succeed it is
important to them to know that they
are the best in their chosen field.
Hard workers, they will be willing to
make many personal sacrifices to
achieve their success. High scorers
require the respect and admiration of
those they perceive as ‘worthwhile
people’. They will routinely set
themselves difficult targets, finding
greatest satisfaction from succeeding
at the most difficult tasks.

Low scorers are not overly
concerned with being the best. Not
particularly desiring the respect and
admiration of others they will not let
career ambitions interfere with
personal and family life. Low scorers
will tend to set themselves realistic
career targets which they believe
they can achieve without too much
difficulty.

ECONOMIC STATUS

High scorers on the need for
economic status tend to desire the
trappings of wealth and the status
which comes with it. Materialistic,
they believe in the pursuit of mater-
ial wealth for the status they regard
ensues from such wealth.

Low scorers would probably see
little point in pursuing wealth after
they had achieved what was, for
them, a comfortable lifestyle. Not
excessively materialistic they would
not be overly impressed by sheer
accumulation of wealth, regarding
an individuals status as something
much more than the amount of
material possessions they had
managed to gain.

SECURITY/SAFETY

High scorers on security tend to be
cautious, safety-conscious people.
They have no particular inclination
to take risks and find no excitement
at all in thoughts of dangerous
pursuits. Preferring a fairly
predictable, routine life they have no
great love of variety.

For low scorers variety is the spice
of life and they are at their happiest
when some risk is involved in their
activities. Tending to enjoy adventur-
ous pursuits they wish to enjoy life to
the full, experiencing as many differ-
ent aspects as they can.

AESTHETICS

High scorers appreciate cultural
activities such as art, music and liter-
ature. They would tend to believe
that artistic, cultural pursuits are
worth following for their own sake.
Intellectually abstract they quite
enjoy discussing issues which would
seem to have little bearing on every-
day life.

Low scorers have little interest in
artistic or cultural pursuits, having
little understanding of what others
see in such topics. Much preferring to
discuss concrete issues they will have
little time for what they would
regard as ‘airy-fairy’ abstract issues.

bm
MORAL VALUES

For high scorers truthfulness and
personal integrity are of highest
importance in living ones life.
Having a belief in basic principles of
right and wrong, they tend to
measure their own, and others,
actions in terms of these fundamen-
tal principles. Thus, high scorers
would conduct their affairs in line
with a strict code of moral values
and expect those around them to do
the same.

Low scorers on moral values do
not particularly believe in a funda-
mental set of principles which dictate
the way one should live ones life.
With no rigid moral code to guide
them they will be more inclined to
view their own, and others, behav-
iour in the light of the circumstances
at the time.

TRADITIONAL VALUES

High scorers on traditional values
tend to have great respect for
authority believing that rules and
laws are meant to be obeyed and not
broken. They will be inclined to
believe that the status quo is to be
maintained and be firm defenders of
all that is traditional. High scorers
will also tend to place emphasis on
national pride, believing that patrio-
tism and loyalty are qualities to be
desired in all citizens.

Low scorers, on the other hand,
will be more inclined to challenge
existing authority, believing that
changes to existing laws and rules
are not only possible, but desirable.
Believing that one should follow the
spirit, rather than the letter, of the
law such people will show little hesi-
tation in breaking an existing rule if
they think the rule unjust.

INDEPENDENCE

High scorers on independence will
believe in standing up for their own
views regardless of what others
might think. Individualists, they will
be very suspicious of anyone in a
position of authority and be fairly
unwilling to submit to such author-
ity. Being totally committed to their
own viewpoint they will treat the
views of others with some suspicion,
being on their guard against
attempts to persuade them to adopt
a different position.

Low scorers are not particularly
concerned with putting their own
views across, being fairly content to
let others have their way. Tending to
believe that those in authority are
just doing their job they will gener-
ally accept the rulings of such people
quite happily.

ETHICAL VALUES

High scorers have a tendency to
believe in some ‘higher-order’ expla-
nation for the world around them
rather than accept a materialistic,
scientific explanation. They will be
reluctant to accept a totally scientific
explanation of the world, but will
instead believe that there are areas of
existence which can only be under-
stood through faith. High scorers may
well feel justified in ignoring rules and
regulations which they believe are
contrary to their particular faith.

Low scorers will tend to believe
that their are scientific, rational
explanations for all phenomenon.
They will assume that even events
which might seem unexplainable at
present will one day be explained
when the required scientific
advances are made. Low scorers will
tend to have little time for mystical,
or religious explanations preferring
to rely instead on rational, logical
arguments.

INTRINSIC VALUES

3PSYCHOMETRIC
PROPERTIES OF THE
VMI
This chapter will present details con-

cerning the psychometric properties
of the Values & Motives Inventory.

The aim will be to show that the VMI
fulfils various technical requirements,
in the areas of standardisation, relia-

bility and validity, which ensure the
psychometric soundness of the test

1 INTRODUCTION

2 GENDER DIFFERENCES ON THE VMI

3 RELIABILITY OF THE VMI

4 VALIDITY

bo INTRODUCTION
In order to provide meaningful inter-
pretations, the VMI was standardised
against a number of relevant groups.
The constituent samples which make
up the VMI norm base are fully
described later.

STANDARDISATION :
NORMATIVE

Normative data allows us to compare
an individuals score on a standard-
ised scale against the typical score
obtained from a clearly identifiable,
homogeneous group of people.0

Standardisation ensures that the
measurements obtained from a test
can be meaningfully interpreted in
the context of a relevant distribution
of scores. Another important techni-
cal requirement for a
psychometrically sound test is that
the measurements obtained from
that test should be reliable.

RELIABILITY

The property of a measurement
which assesses the extent to which
variation in measurement is due to
true differences between people on
the trait being measured or to
measurement error.

Reliability is generally assessed
using two specific measures, one
related to the stability of scale scores
over time, the other concerned with
the internal consistency, or homo-
geneity of the constituent items that
form a scale score.

RELIABILITY : STABILITY

Also known as test-retest reliability,
an assessment is made of the similar-
ity of scores on a particular scale
over two or more test occasions. The
occasions may be from a few hours,
days, months or years apart.
Normally Pearson correlation coeffi-
cients are used to quantify the
similarity between the scale scores
over the two or more occasions.

Stability coefficients provide an
important indicator of a test’s likely
usefulness of measurement. If these
coefficients are low (< approx. 0.6) then it is suggestive of either that the behaviours/attitudes being measured are volatile or situationally specific, or that over the duration of the retest interval, situational events have the content of the scale irrelevant or obsolete. Of course, the duration of the retest interval provides some clue as to which effect may be causing the unreliability of measurement. However, the second measure of a scales reliability also provides valu- able information as to why a scale may have a low stability coefficient.

bp

RELIABILITY : INTERNAL
CONSISTENCY

Also known as scale homogeneity, an
assessment is made of the ability of
the items in a scale to measure the
same construct or trait. That is a
parameter can be computed that
indexes how well the items in a scale
contribute to the overall measure-
ment denoted by the scale score. A
scale is said to be internally consis-
tent if all the constituent item
responses are shown to be positively
associated with their scale score.

The most common measure of
internal consistency is Cronbach’s
Alpha. If the items on a scale have
high inter-correlations with each
other, and with the total scale score,
then coefficient alpha will be high.
Thus a high coefficient alpha indi-
cates that the items on the scale are
measuring very much the same
thing, while a low alpha would be
suggestive of either scale

items

measuring different attributes or the
presence of error.

The fact that a test has high inter-
nal consistency and stability
coefficients only guarantees that it is
measuring something consistently. It
provides no guarantee that the test is
actually measuring what it purports
to measure, nor that the test will
prove useful in a particular situation.
Questions concerning what a test
actually measures and its relevance
in a particular situation are dealt
with by looking at the tests validity.

VALIDITY

The ability of a scale score to reflect
what that scale is intended to
measure. Kline’s (1993) definition is
“A test is said to be valid if it
measures what it claims to measure”.

Reliability is generally investigated
before validity as the reliability of test
places an upper limit on tests validity.
It can be mathematically demon-
strated that a validity coefficient for a
particular test can not exceed that
tests reliability coefficient.

VALIDITY : CONSTRUCT
VALIDITY

Construct validity assesses whether
the characteristic which a test is
actually measuring is psychologically
meaningful and consistent with the
tests definition.

Validation studies of a test investi-
gate the soundness and relevance of
a proposed interpretation of that test.
Two key areas of validation are
known as criterion validity and
construct validity.

VALIDITY : CRITERION
VALIDITY

Criterion validity involves translating
a score on a particular test into a
prediction concerning what could be
expected if another variable was
observed.

The criterion validity of a test is
provided by demonstrating that
scores on the test relate in some
meaningful way with an external
criterion. Criterion validity comes in
two forms –predictive and concur-
rent. Predictive validity assesses
whether a test is capable of predict-
ing an agreed criterion which will be
available at some future time –e.g.
can a test predict the likelihood of
someone successfully completing a
training course. Concurrent validity
assesses whether the scores on a test
can be used to predict a criterion
measure which is available at the
time of the test –e.g. can a test
predict current job performance.

The construct validity of a test is
assessed by demonstrating that the
scores from the test are consistent
with those from other major tests
which measure similar constructs
and are dissimilar to scores on tests
which measure different constructs.

bq
Table 1 lists mean score differences between 68 male and 87 female students
respondents drawn from student populations. Of the 11 value scales of the
VMI, four reveal statistically significant differences between male and female
respondents. The most substantial difference is found on the Financial Status
scale where a difference equivalent to about half the pooled SD is observed,
with the males mean rating higher than of females. Females rate Moral and
Ethical values lower than the males but curiously males appear to endorse
traditional values more than their female counter-parts. It should be noted
that while these constitute statistically significant differences, the absolute
differences remain small and with the possible exception of the Financial
Status value, are unlikely to have ramifications as far as profile interpretation
is concerned. The question as to the Financial Status scale would be to ascer-
tain whether the difference observed reflects a true difference in the
population or is an artefact of the test.

RELIABILITY OF THE VMI
Internal consistency reliabilities (Cronbach’s Alpha) were computed on the
total standardisation sample. The coefficients are computed over the
combined males & females. (see Table 2).

With the exception of the Achievement sub-scale of the VMI, the scales
approximate or exceed acceptable levels of internal consistency. Eight of the
11 VMI Value scales exceed the 0.7 level, considered to reflect acceptable
measurement error. The Moral and Independence scales fall slightly below
(at 0.68 and 0.66 respectively) although the Achievement scale, with a
Standard Error of Measurement equivalent to almost 1.4 sten points, should
be treated with a greater degree of caution. Additional items are currently in
the process of being trialled and this scale will be updated once the data is
forthcoming. Reliability estimates for the response style indicators Social
Desirability and Infrequency are provided as these are dedicated scales
unlike the Central tendency and A

cq

uiescence scales which are simply a sum
of all central and acquiescent responses. The estimate for SD is acceptable at
0.64 although the Infrequency scale falls somewhat short of ideal. This is in
part due to its distribution which is far from normal.

GENDER DIFFERENCES ON
THE VMI

br

VMI Scale

Affiliative
Altruistic
Affection
Achievement
Financial
Security
Aesthetics
Moral
Traditional
Independence
Ethical
Social Desirability
Infrequency
Central Tendency
Acquiescence

VMI Scale

Affiliative
Altruistic
Affection
Achievement
Finance
Security
Aesthetics
Moral
Traditional
Independence
Ethical
S.D.
Infrequency

Table 1: Mean raw score differences & t-test significance levels by gender.

Mean F

35.1

8

43.98
36

.44

25.43
39.72
14.97
39.54
21

.

26

25.

20

19.08
21

.68

20

.33

.66
110.

25

55.38

Mean M

36.46
41.93
36.8

1

26.21
44.7

9

13.66
39

.

16

22.75
27.

07

19

.

28

20

.

32

21

.53

.63
111.66
54

.28

SD F

6

.32

6

.47

6.

29

3.70
8.33
4

.35

6.33
3.63
5

.

24

3.

12

3.51
4.35
.85

24.87
10.66

SD M

5.52
6.64
4.

49

3

.

40

6.

01

4.78
7.85
3.34
5

.

14

2.65
4

.

19

4.28

.71

23.69
8.90

t-value

-1.31
1.94
–

.41

-1.35
-4.

23

1.77
.33

-2.62
-2.23

-.42

2.19

-1.71
.18

-.36

.68

p-value

.19

11

.0547

.68

05

.1790

.00

00

.0782

.7400

.0097

.0270

.6748

.0300

.0890

.8585

.72

13

.4947

Table 2: VMI Internal Consistencies & Item Total Correlations (ITC’s)

items

11
12

10

7

12
6

11
7
9
6
7
9
7

Alpha

0.74
0.74
0.79
0.53
0

.83

0.79
0.83
0.68
0.70
0.66
0.70
0.64
0.52

ITC

.

22

.19

.28

.14

.30

.40

.32

.24

.22

.24

.26

.16

.

15

SEm

1.

02

1.02
0.92
1.37
0.82
0.92
0.82
1.13
1.10
1.

17

1.10
1.20
1.39

bs VALIDITY
As was said in the introduction once
we have ascertained the reliability of
a test we must address its validity. It
is important to know that the
constructs we are measuring are
valid, that it is indeed measuring the
characteristic it purports to measure.
This section of the manual provides
a number of tables which demon-
strate that the dimensions of the VMI
are consistent with similar measures.

INTER-CORRELATIONS OF
VMI SCALES

A sample of 159 Psychology and
MBA’s students completed the VMI
as part of an introductory course in
Personnel Assessment. Table 3
provides the full VMI inter-correla-
tion matrix.

Generally the inter-correlations
are fairly modest, with a range from
0 to 0.57, with a low median value
of 0.10. This indicates that the VMI
scales are generally independent of
each other and are measuring
distinct aspects of the individual’s
value system. The highest correla-
tions are found between Affection
and Affiliation (0.57), Traditional
and Moral (.49) Financial Status
and Altruism (-.46) and finally
Affection and Altruism (.39). Two
response style indicators,
Acquiescence and Central Tendency
are also highly correlated (.57) but
this is simply related to the fact that
they share a significant number of
items.

bt
1

1

26

57

–

09

02

00

–

04

07

10

-08

05

04

-11

03

04

2

26

1

39

–

06

-46

06

24

27

-10

-02

11

-02

-10

-27

23

3

57

39

1

-12

-09

05

-01

15

05

-15

05

-03

-06

-16

28

4

-09

-06

-12

1

40

-03

-01

22

14

-09

05

12

04

-29

25

5

02

-46

-09

40

1

-21

-13

03

19

-17

-21

-07

11

-07

01

6

00

06

05

-03

-21

1

-10

17

02

-03

19

-04

-02

16

-01

7

-04

24

-01

-01

-13

-10

1

04

-06

17

-06

-09

-06

-24

04

8

07

27

15

22

03

17

04

1

49

-15

32

24

-01

-06

23

9

10

-10

05

14

19

02

-06

49

1

-30

27

24

-00

09

09

10

-08

-02

-15

-09

-17

-03

17

-15

-30

1

-03

-10

12

-17

10

11

05

11

05

05

-21

19

-06

32

27

-03

1

26

05

02

20

12

04

-02

-03

12

-07

-04

-09

24

24

-10

26

1

05

15

13

13

-11

-10

-06

04

11

-02

-06

-01

-00

12

05

05

1

-21

19

14

03

-27

-16

29

-07

16

-26

-06

09

-17

02

15

-21

1

-57

15

04

23

28

25

01

-01

04

23

09

10

20

13

19

-57

1

10 Indep Independence
12 SocD Social Desirability
14 Cent Central Tendency

Table 3: VMI Inter-correlation Matrix

VMI

1 Affiliation

2 Altruism

3 Affection

4 Achievement

5 Finanical

6 Safety

7 Aesthetics

8 Morality

9 Tradition

10 Indep

11 Ethics

12 Social D

13 Infrequency

14 Central T

15 Aquiescence

ck
RELATIONSHIP BETWEEN VMI & MAPP

A sample of 59 Psychology under-graduates volunteered to complete the VMI
and MAPP as part of a validation study in exchange for feedback on their
results.

From Table 4, we can observe a large number of psychologically meaning-
ful correlations between the VMI and MAPP, specifically in those value areas
which both instruments cover. A remarkably high correlation is registered
between MAPP Material Wealth and VMI Financial (.83). This suggests that
these two scales are inter-changeable. VMI Financial also correlates strongly
with MAPP Personal Authority (.71), Competition (.5) and negatively (-.55)
with MAPP Altruism. VMI Altruism also converges with its MAPP namesake
(.71) and registers moderate to high correlations with Intimacy (.49) and
Personal Authority (-.52). Both VMI Affiliation and Affection register a 0.68
correlation with MAPP Intimacy, suggesting MAPP Intimacy may be tapping
into aspects of these two VMI scales. VMI Need for Achievement correlates
highly with a number of MAPP scales, Competition (.48), Responsibility
(.52), Personal Authority (.56) and Work (.53), without a single MAPP scale
standing out. VMI Safety which measures the degree to which an individual
places emphasis on personal security and harm-avoidance, correlates nega-
tively with MAPP Novelty (-.50), but only registers a .39 correlation with
MAPP Security. This is likely to be due to the different focus of these same
name scales. MAPP in contrast to VMI is more directed at job security and
long-term future, with no reference to danger and risk-taking. VMI Aesthetics
is related to MAPP Self-Expression (.45), Novelty (.41) and Intellect (.35)
which reflects aspects of this scale.

VMI intrinsic scales have no clear MAPP counter-parts and conse-
quently only register modest correlations with related MAPP scales. VMI
Traditional hardly registers with MAPP at all with only one correlation (with
Personal Authority) above the .3 level. Equally, VMI Moral Values correlates
positively (.33) with Altruism and negatively with Levity (-.30) although this
may be related to a social desirability effect (VMI S.D. correlates with both
these MAPP scales). VMI Ethics has similarly low correspondence with
MAPP with two -.33 correlations with Material Wealth and Levity. Finally
VMI Independence is negatively related to Competition (-.44), Personal
Authority (-.45) and also registers a .35 with MAPP Self-expression.

Of the VMI response style indicators, Central Tendency correlates nega-
tively with Responsibility (-.30), Novelty (-.39) and Self-expression (-.36),
suggesting that those who emphasise value each of these three values are
more likely to avoid the central responses.

cl

Tradition
Morality
Independence
Ethical
Altruism
Affiliation
Affection
Achievement
Financial
Safety
Aesthetic
Infrequency
Social Desirability
Central Tendency
Acquiescence

Table 4: Correlations between VMI and MAPP

Mw

-.30

-.33

-.43

.28

.83

-.30

Cpt

-.44

-.42

.48

.50

-.27

Res

-.27

-.32
.52
.36

Rcg

-.36

.41

.47

.28

Per

.33

–

.45

-.52

.56

.71

Rsp

-.28

.40

.44

-.30

Int

-.28

.35

Nov

-.50
.41

–

.39

SlfE

.35

.26

.45

-.36

Alt

.33

.71

.41

.28

-.55

.25

.29

Inti

.49

.68

.68

.32

-.29

Lev

-.29
-.30

-.33

.25

-.35

-.35

-.27

Sec

.30

.39

.39
-.34

Wor

.53

.33

Mw Material Wealth Nov Novelty
Cpt Competition SlfE Self-Expression
Res Results Alt Altruism
Rcg Recognition Inti Intimacy
Per Personal Authority Lev Levity
Rsp Responsibility Sec Security
Int Intellect Wor Work

cm
RELATIONSHIP BETWEEN VMI & 16PF

A sample of 100 MBA students attending a prestigious London Business
School completed both the VMI and the 16PF version 5 as part of an intro-
duction to personnel assessment methods.

Although typically the correlations between the VMI and 16PF version 5
are modest in magnitude, they are nonetheless generally psychologically
meaningful, given that the two tests are in fact measuring different personal
characteristics.

The VMI interpersonal values generally correlate with the 16PF extrover-
sion factors. VMI Affiliative is related strongly to Group Dependence (Q2)
but also taps into aspects of Apprehension (O). Affection registers a similar
pattern of correlations with 16PF although there appears to more congruence
with Warmth (A) and low Privateness (N). VMI Altruism relates primarily to
Rule Consciousness (G) which would suggest that there may be some degree
to which endorsement of Altruism is an aspect of conformity.

VMI Achievement and Finance are clearly not directly measured by 16PF-
5, although the former correlates .33 with Perfectionism which reflects one
aspect of Achievement orientation. Aesthetics relates to a number of 16PF-5
factors, including Reasoning (B), Sensitivity (I) and Openness to Change. In
addition, it registers correlations of above 0.3 with Dominance and Social
Boldness suggesting that those who value Aesthetics may be outspokenly so.
VMI Moral and Traditional scales both relate to 16PF Rule Consciousness
(G) and in addition, Traditional registers a small negative correlation with
16PF Reasoning. Finally, VMI Ethical, which assesses the importance placed
upon ‘higher’ forces in determining one’s future, this registers very small
correlations with Submissiveness (E) and Apprehension (O).

Of the Response Style indicators the corresponding measures of Social
Desirability correlate only modestly at 0.4. VMI SD correlates almost as
highly (-.37) with 16PF-5 Tension which is perhaps best explained by the
item content of the latter e.g. I am happy to wait in queues!

cn
VMI Scales 16PF-5 Scales

Affiliative A: Warmth .25, F: Liveliness .28, N: Privateness -.35,
O: Apprehension .37, Q2: Self-reliance -.52

Altruistic G: Rule-Consciousness .36, N: Privateness -.25,

Affection A: Warmth .29, N: Privateness -.41, O: Apprehension .36,
Q2: Self-reliance -.28

Achievement Q3: Perfectionism .33

Finance Q4: Tension .24

Security E: Dominance -.25, F: Liveliness -.25, Q1: Openness to Change -.27

Aesthetics B: Reasoning .28, E: Dominance .35, H: Social-Boldness .30,
I: Sensitivity .42, Q1: Openness to Change .39

Moral G: Rule-Consciousness .57

Traditional B: Reasoning -.28, G: Rule-Consciousness .38

Independence B: Reasoning .29, H: Social-Boldness -.29, N: Privateness

Ethical E: Dominance -.25, O: Apprehension .24

Social Desirability G: Rule-Consciousness .28, Q4: Tension -.37,
IM: Impression Management .40

Central Tendency E: Dominance .20

Acquiescence O: Apprehension .23

Table 5: Correlations between VMI and 16PF Form 5

co
RELATIONSHIP BETWEEN VMI & OPP

A sample of 59 Psychology under-graduates volunteered to complete the VMI
and MAPP as part of a validation study in exchange for feedback on their
results.

A number of notable correlations were found between OPP and VMI,
suggesting a fair degree of overlap in certain areas. Two correlations in excess
of 0.7 were found between OIP and OPP, suggesting near equivalence in
measurement focus. These were between VMI Affiliation and OPP
Gregarious and between VMI Aesthetics and OPP Abstract-Pragmatic. This
raises the issue of whether in some cases, measures of personality traits and
values are in fact tapping into the same underlying psychological constructs.
VMI Altruism correlates highly with OPP Trusting and Social Desirability
(which doubles up as a measure of Conformity). This would suggest, as
noted previously, that VMI Altruism may be subject to impression manage-
ment. Those scoring high on VMI Affection tend to have higher scores on
OPP Gregarious, Emotional and Trusting, although none of these correla-
tions exceed 0.5. In the VMI Extrinsic domain, Safety, Financial and
Achievement Status barely correlate with OPP scales. In contrast, the VMI
intrinsic scales do find some modest congruence with equivalent OPP dimen-
sions. VMI Moral, correlates with OPP SD (Conformity), Trusting and
Rigidity. VMI Traditional correlates negatively with OPP Flexibility and
whereas VMI Independence correlates inversely with the same. Finally, VMI
Ethical registers only modest, but meaningful correlations with OPP Rigidity,
Emotionality, Genuineness, and Externality (a form of fatalism). In the
response-style arena, the respective measures of social desirable and central
responding correlate 0.58 and .75 respectively, and interestingly, Central
Tendency also correlates -.50 with Flexibility.

cp

Affiliation
Altruism
Affection
Achievement
Finance
Safety
Aesthetics
Moral
Traditional
Independence
Ethical
Distortion
Infrequency
Central Tendency
Acquiescence

Table 6: Correlations between VMI and OPP

Ass Fle Tru Phl Gre Per Con Ext Pra Con Mid

.37 .73 -.25 .27

.51 .37

.31 -.27 .47
.30 -.37 .27 .35
.33 -.25 -.32 .27

-.33
.33 .32 -.71 .26

-.32 .34 .46
-.32 -.58 .25 .25

.52 -.27 -.32
-.32 -.32 -.32 .32
-.29 .27 .58

-.29 -.27
-.30 -.50 .37 .32 .75

-.40

Ass Empathic-Assertive Con Composed-Contesting
Fle Detailed-Flexible Ext Optimistic-Pessimistic
Tru Cynical-Trusting Pra Abstract-Pragmatic
Phl Emotional-Phlegmatic Con Social-Desirability
Gre Reserved-Gregarious Mid Central Tendency
Per Genuine-Persuasive

cq

4ADMINISTRATION
INSTRUCTIONS
BEFORE STARTING THE QUESTIONNAIRE

Put candidates at their ease by giving information about yourself, the purpose
of the questionnaire, the timetable for the day, if this is part of a wider assess-
ment programme, and how the results will be used and who will have access
to them. Ensure that you and other administrators have switched off mobile
phones etc.

The instructions below should be read out verbatim and the same script
should be followed each time the VMI is administered to one or more candi-
dates. Instructions for the administrator are printed in ordinary type.
Instructions designed to be read aloud to candidate incorporate a grey shaded
background, italics and speech marks.

If this is the first or only questionnaire being administered give an introduc-
tion as per or similar to the following example:

“From now on, please do not talk among yourselves, but
ask me if anything is not clear. Please ensure that any
mobile telephones, pagers or other potential distractions are
switched off completely. We shall be doing the Values &
Motives Inventory which has no time limit, however most
people take about 20 minutes. During the test I shall be
checking to make sure you are not making any accidental
mistakes when filling in the answer sheet. I will not be
checking your responses.”

WARNING: It is most important that answer sheets do not go astray. They
should be counted out at the beginning of the test and counted in again at the
end.

cs
Continue by using the instructions EXACTLY as given. Say:

DISTRIBUTE THE ANSWER SHEETS

Then ask:

“Has everyone got two sharp pencils, an eraser, some rough
paper and an answer sheet.”

Rectify any omissions, then say:

“Print your surname, first name and title clearly on the line
provided, followed by your age and sex. Please insert
today’s date which is [ ] on the ‘Comments’ line”

Walk around the room to check that the instructions are being followed.

WARNING: It is vitally important that test booklets do not go astray. They
should be counted out at the beginning of the session and counted in again at
the end.

DISTRIBUTE THE BOOKLETS WITH THE INSTRUCTION:

“Please do not open the booklet until instructed.”

Remembering to read slowly and clearly, go to the front of the group and say:

“Please open the booklet and follow the instructions for this
test as I read them aloud.” (Pause to allow booklets to be
opened).

This is a questionnaire concerning your interests, prefer-
ences and feelings about a range of things.

You are asked to rate yourself on a scale from 1 to 5 on
each question. When you have chosen the answer appropri-
ate for YOU, record this by blackening the corresponding
box on the answer sheet.

For example:

Ratings:

1 2 3 4 5
Strongly Agree In Disagree Strongly

Agree between Disagree

1. I like to watch the news on TV.

ct
If you strongly agreed with this statement, you would fully
blacken box 1 against question 1 on your answer sheet.”

Check for understanding of the instructions so far, then say:

“When answering the questions please remember the
following:

1. Do not spend too much time pondering over the answer
to each question. The information given in a question
may not be as full as you would wish, but answer as
best you can.

2. Please try to avoid the middle (In between) answer
wherever possible.

3. Be as honest and truthful as you can. Don’t give an
answer just because it seems to be the right thing to say.

4. Make sure you answer every question, even those which
do not seem to apply to you.

5. If you wish to change an answer, please erase it and
insert your new answer.”

Then say very clearly:

“Is everybody clear about how to do this test?”

Deal with any questions appropriately, then say:

“Please begin”

Answer only questions relating to procedure at this stage, but enter in the
Administrator’s Test Record any other problems which occur. Walk around
the room at appropriate intervals to check for potential problems. When
everybody has completed the questionnaire:

COLLECT ANSWER SHEETS & TEST BOOKLETS, ENSURING THAT
ALL MATERIALS ARE RETURNED (COUNT BOOKLETS & ANSWER
SHEETS)

Then say:

“Thank you for completing the Values and Motives
Inventory.”

dk

5REFERENCES

Allport, G. W. et al (1960) Study of
Values. Manual & Test Booklet.
Boston: Houghton Mifflin.

Budd, R. J. (1991). Manual for the
OPP. Letchworth, Herts: Psytech
International.

Budd, R. J. (1991). Manual for the
JTI. Letchworth, Herts: Psytech
International.

Campbell, D. P. (1971). Handbook
for the Strong Vocational Interest
Blank. Stanford: Stanford University
Press.

Costa, P. T. & McCrae, R. R, (1992)
Manual for the Neo PI-R. Odessa,
Florida. Psychological Assessment
Resources.

Cronbach, L.J. (1951). Coefficient
alpha and the internal structure of
tests. Psychometrica, 16, 297-334

Hunter, R A & Roberts, A. M (1989)
Manual for the Managerial and
Profesional Profiler. Lewes, Sussex:
Knight Chapman Psychological.

Kline, P. (1993). Personality: The
Psychometric View. London,
Routledge.

Rokeach, M (1967) Value Survey.
Sunnyvale, CA: Halgren tests

Rokeach, M (1973) The Nature of
Human Values. New York: Free
Press.

Russell, M & Darcie K. (1994)
Manual for the 16PF Fifth Edition.
Champaign, Illinois: Institute for
Personality and Ability Testing.

Criteria Ratings Points

Content

Types of
Reliability
& Validity

7 to >6.

0 pts

Advanced

The types of reliability
and validity are clearly
explained. Support from
reading is included.

6 to >4.0 pts

Proficient

The types of reliability and
validity are reasonably
explained. Support from
reading is mostly included.

4 to >0.0 pts

Developing

The types of reliability and
validity are minimally
addressed. Limited or no
support from reading.

0 pts

Not
Present

7 pts

Content

Reliability
Explored

7 to >6.0 pts

Advanced

The proper categories of
low, acceptable, and high
ranges of alpha
coefficients are clearly
identified. The number of
scales in each category is
stated. Conclusions
drawn, based on the
number of scales with
low, acceptable, and high
alpha coefficients, are
included.

6 to >4.0 pts

Proficient

The proper categories of
low, acceptable, and high
ranges of alpha coefficients
are reasonably identified.
The number of scales in
each category is mostly
stated. Conclusion drawn,
based on the number of
scales with low,
acceptable, and high alpha
coefficients are mostly
included.

4 to >0.0 pts

Developing

The proper categories of
low, acceptable, and high
ranges of alpha coefficients
are minimally addressed.
The number of scales in
each category is not clearly
stated. Conclusion drawn,
based on the number of
scales with low, acceptable,
and high alpha coefficients,
are not clearly addressed.

0 pts

Not
Present

7 pts

Content

VMQs
Validity
Explored

7 to >6.0 pts

Advanced

Comparison between the
VMQ population and the
populations of the tests
used to evaluate validity
is clearly stated. Issues
raised when evaluating
validity based on the
population traits of
coefficients are included.

6 to >4.0 pts

Proficient

Comparison between the
VMQ population and the
populations of the tests
used to evaluate validity is
reasonably stated. Issues
raised when evaluating
validity based on the
population traits of
coefficients are mostly
included.

4 to >0.0 pts

Developing

Comparison between the
VMQ population and the
populations of the tests used
to evaluate validity is
minimally addressed. Issues
raised when evaluating
validity based on the
population traits of
coefficients are not clearly
stated.

0 pts

Not
Present

7 pts

Content

Norming
Population

7 to >6.0 pts

Advanced

How sample size and
nature of population may
influence the constructs
purportedly measured by
the test is clearly
articulated. Strengths and
weaknesses in these
areas are included.

6 to >4.0 pts

Proficient

How sample size and
nature of population may
influence the constructs
purportedly measured by
the test is reasonably
articulated. Strengths and
weaknesses in these areas
are mostly included.

4 to >0.0 pts

Developing

How sample size and nature
of population may influence
the constructs purportedly
measured by the test is
minimally articulated.
Strengths and weaknesses
in these areas are not
clearly stated.

0 pts

Not
Present

7 pts

Benchmark Exploring Reliability and Validity Grading Rubric |
COUC521_B05_202240

Criteria Ratings Points

Content

Opinion

7 to >6.0 pts

Advanced

Formed opinion about the
quality of the test using
the information conveyed
in the report is clearly
stated.

6 to >4.0 pts

Proficient

Formed opinion about the
quality of the test using the
information conveyed in the
report is reasonably stated.

4 to >0.0 pts

Developing

Formed opinion about the
quality of the test using the
information conveyed in the
report is not clearly stated.

0 pts

Not
Present

7 pts

Structure

Paper:
Spelling &
Grammar

8 to >6.0 pts

Advanced

Spelling and grammar
are correct. Sentences
are complete, clear, and
concise. Paragraphs
contain appropriately
varied sentence
structures.

6 to >4.0 pts

Proficient

Spelling and grammar have
some errors. Sentences
are presented as well.
Paragraphs contain some
varied sentence structures.

4 to >0.0 pts

Developing

Spelling and grammar errors
distract. Sentences are
incomplete or unclear.
Paragraphs are poorly
formed.

0 pts

Not
Present

8 pts

Structure

Paper:
Length &
APA
Formatting

7 to >6.0 pts

Advanced

Paper is 600-900 words
in length (not including
the title and reference
pages). Paper is
formatted in APA
including font, title page,
margins, and section
headings. Where
applicable, references
are cited in current APA
format.

6 to >4.0 pts

Proficient

Paper is 450-599 words in
length. Paper is mostly
formatted in APA, including
font, title page, margins,
and section headings.
Where applicable,
references are cited with
nearly accurate APA
formatting.

4 to >0.0 pts

Developing

Paper is 449 words or less
in length. Paper is not
accurately formatted in APA,
including font, title page,
margins, and section
headings. Where applicable,
references are minimally or
not cited in current APA
format.

0 pts

Not
Present

7 pts

Total Points: 50

Benchmark Exploring Reliability and Validity Grading Rubric |
COUC521_B05_202240