Researcher are often confused with the idea of Reliability and Validity, but the terms actually describe two completely different concepts, although they are often closely inter-related. This distinct difference is best summed up with an example:

https://explorable.com/definition-of-reliability

• Reliability is an essential component of validity however, on its own, is not a sufficient measure of validity. A test can be reliable but not valid, whereas a test cannot be valid yet unreliable.

• Reliability, in simple terms, describes the repeatability and consistency of a test. Validity defines the strength of the final results and whether they can be regarded as accurately describing the real world.

https://explorable.com/definition-of-reliability

https://explorable.com/definition-of-reliability

In order to ensure effectiveness in the gathering of data needed in the research, the instruments that will be used by the researcher in conducting the study should be reliable and valid. Webster Dictionary (1980) defines reliability as “the state of being reliable”, hence, that may be relied or depended upon; it is the degree of being precise and consistent. Precision and consistency should be the qualities of the instrument to be considered to be reliable. Hence, consistency does not guarantee reliability

A sphygmomanometer may give the same measurement of blood pressure, but it may not be giving the true and accurate measure. It may not be a reliable instrument when used to get the blood pressure. Thus, it is necessary compare this with other standardized instruments to make it consistent and accurate.

Therefore, if the data obtained are

close to what we feel we are measuring, then, we say that the instrument is valid.

Validation Procedure

A valid instrument gathers all the data that are needed for the researcher to be able to adequately address the research problem (Espero, 1999). Further, content validity measures the representativeness of the items as they address the variables of interest (Caccam, 1999).

Both the questionnaire checklists were pretested for validity and reliability. To enhance the content validity of the questionnaire checklist, five evaluators were requested to assess the items in the questionnaire checklists adequately provide information or data to meet requirements of the problems under study.

The result of the evaluators’ assessment on the validity of the two sets of questionnaire checklists were considered by the researcher to be suitable for the purpose of generating the needed data to meet the requirements of the research problem.

The questionnaires were pilot-tested to a few respondents (Commercial/Industrial/Residential) in Dagupan City. Comments and suggestions by the respondents were used to improve the item-questions. After which, the content validity of the improved questionnaires were determined by submitting them for physical and ocular examination to determine the adequacy and suitability of items to the age and behavioral patterns, vocabulary level and experiential background of the intended respondents or subjects of the study to a pool of five fire experts in the province and the region.

The questionnaire for the community residents was judged by a few residents of Urdaneta City. Computed content validity of the questionnaire for BFP personnel was 4.76 with descriptive rating of Very Highly Valid, while the questionnaire for the residents was given a rating of 4.16 with descriptive equivalent of Highly Valid.

Rating Scale on the Content Validity of the Questionnaire

Point Value

Descriptive Rating Extent of Description

1 Not Valid (NV) At least 20% of the items in the questionnaire are appropriate and relevant. Revision is needed for all the items

2 Least Valid (LV) At least 40% of the items in the questionnaire are appropriate and relevant.

3 Moderately Valid (MV) At least 60% of the items in the questionnaire are appropriate and relevant.

4 Very Valid (VV) At least 80% of the items in the questionnaire are appropriate and relevant.

5 Very Much Valid (VMV

At least 100% of the items in the questionnaire are appropriate and relevant.

Summary of Rating on the Content Validity of the Questionnaire

Rater Mean Rating

1 4.8

2 4.5

3 4.5

4 4.2

5 4.3

Total 22.3

22.3/5 = 4.46

Computation of Point Ranges

High (H) - Low (L) H = 5 – 1 5 = 4/5 = .8

Point Ranges Descriptive Rating 4.21 – 5.00 Very Much Valid (VMV) 3.41 - 4.20 Very Valid (VV) 2.61 – 3.40 Moderately Valid (MV) 1.81 - 2.60 Least Valid (LV) 1.00 – 1.80 Not Valid (NV)

Reliability Index of the Instrument

Items

Rosanna T. Flores

Mary Jane Robles Ben Madirs Badu Rawling Clarissa Abel RJ Bitonio %

Consistency

Respondent 1 Respondent 2 Respondent 3 Respondent 4 Respondent 5 Respondent 6

Pre test Final Test Pre test

Final Test Pre test

Final Test Pre test

Final Test Pre test

Final Test Pre test Final Test

16.66 16.66 16.66 16.66 16.66 16.66 16.66 16.66 16.66 16.66 16.66 16.66 100%

1

Characteristics of cooperative beneficiaries

- type of service

- registry year

- membership

- financial status II Status of Cooperatives

- proposed application

- Amt of loan extended

- Terms of assistance

III Fund Administration

- Selection of member beneficiaries

- Utilization of loans

- Records management

- Loan Collection System

- Visitation of Projects x 83.33

IV Fund Accountability

- M & E in the use of Fund

- Grant Liquidation x x x 50 - compliance to CDA x x 66.68

- repayment Performance

V Problems encountered x x x x 33.33

VI Impact indices

- net Income - ROI

- Services rendered

- Core Mgmt team - Members' Equity

- Membership Outreach

VII Leadership traits of the BOD

- Personality

- Intelligence & ability - Social x x 66.68

-Task related x x x x 33.33

Average Reliability Index 90.12

Reliability Formula Total frequency of responses by total items

responses

= 2,433.31 27

90.12

Cont.

Definition of Reliability

• "Yielding the same or compatible results in different clinical

experiments or statistical trials“

• In normal language, the use of the word reliable mean that something is dependable and that it will give the same outcome every time. We might talk of a football player as reliable, meaning that he gives a good performance game after game.

• Reliability is also extremely important

externally, and another researcher should be

able to perform exactly the same experiment,

with similar equipment, under similar

conditions, and achieve exactly the same

results. If they cannot, then the design is

unreliable.

Compute Cronbach Coefficient Alpha, Kuder Richardson (KR) Formula, or split-half Reliability Coefficient to check for the internal consistency within a single test. Cronbach Alpha is recommended over the other two for the following reasons:Cronbach Alpha can be used for both binary-type and large-scale data. On the other hand, KR can be applied to dichotomously-scored data only. For example, if your test questions are multiple choices or true/false, the responses must be binary in nature (either right or wrong). But if your test is composed of essay-type questions and each question worths 10 points, then the scale is ranged from 0 to 10.

Which reliability coefficients to use?

To ensure the quality of the research instrument, the instrument was validated by ten experts from the CDA, the academe, and the cooperative federation for its content and responsiveness to the intended respondents. Comments from the validators were incorporated improving the contents of the questionnaires.

As to the reliability index of the instrument, the questionnaire was tested in its entirety, while the subscales of the instrument were likewise tested independently. Item analysis was made to determine the internal consistency and reliability of each individual item, as well as each subscale. Cronbach’s coefficient alpha was used to determine the internal reliability of the instrument which is for consistency or coefficient reliability (Cronbach, 1951).

Cronbach alpha is an index of reliability associated with the variation accounted by the true score underlying of “the underlying construct”. It allows the researcher to measure the internal consistency of scale items based on the average inter item correlation.

Indicates the extent to which then items in

your questionnaire are related to each other; indicates whether a scale is unidimensional or multidimensional

http://www.docstoc.com/docs/106429775/Cronbach%E2%80%99s-alpha

Interpreting Scale Reliability

The higher the score the more reliable the generated score is. A score of 70 or greater is generally considered to be acceptable.

.90 or > High Reliability (HR)

.80 -.89 Good Reliability (GR)

.70 -.79 Acceptable Reliability (AR)

.60 - 69 Marginal Reliability (MR)

Lower threshold are sometimes used in the literature

http://www.docstoc.com/docs/106429775/Cronbach%E2%80%99s-alpha

Reliability for a procedure is essential before its validity can be considered and the actual reliability sets the ceiling for the maximum validity the instrument can possess (Fox, 1969).

In thesis and dissertation writing, the validity and reliability of the instruments could be done by repeated dry-runs. As the saying goes, “Practice makes perfect”.

References

Bitonio, Josefina B. The Administration of and Accountability for Countrywide Development Fund for Cooperative Development, A Dissertation (2002) University of Luzon.

Gandia, Ernesto F. Fire Prevention Program (FPP) of the Bureau of Fire Protection (BFP) in Urdaneta City. A Masteral Thesis (2008) Pangasinan State University

http://www.docstoc.com/docs/106429775/Cronbach%E2%80%99s-alpha

https://explorable.com/definition-of-reliability