Good Practices for Quality Assurance Reviewers: Audit Sampling Planning, Documentation, and Reporting

Good Practices for Quality

Assurance Reviewers: Audit

Sampling Planning,

Documentation, and Reporting

June 2021

Good Practices for Quality Assurance Reviewers: Audit Sampling Planning, Documentation, and Reporting

June 2021

EXECUTIVE SUMMARY

Objective

The purpose of this white

paper is to share good

practices related to

performing quality

assurance reviews of

audit evidence obtained

from sampling.

Approach

One of QAWG’s goals is

to identify and document

good practices to help the

OIG community improve

QA functions. To

implement this goal,

QAWG, through FAEC,

sent a survey in July 2019

to senior OIG audit

leadership and managers

to identify key areas of

concern about the

application or

interpretation of

performance audit

standards. Sampling was

identified as an area of

concern. The QAWG

formed a task team to

identify and summarize

good practices for

performing a QA review

of audit evidence

obtained from sampling.

This white paper provides good practices for performing a QA review of audit evidence

obtained from sampling, as well as good practices for documenting audit sampling. To

develop the good practices, QAWG reviewed various OIGs’ sampling policies and

procedures; the generally accepted government auditing standards (GAGAS), also

known as the Yellow Book (2018 edition); GAO’s Using Statistical Sampling manual

(1992); and American Institute of Certified Public Accountants AU-C Section 530

Sampling Plan Procedures.

Audit sampling is the selection and evaluation of less than 100 percent of the

population, which the auditor expects to be representative of the population and will

provide a reasonable basis for conclusions about the population. This process allows the

audit team to gain an understanding of characteristics of the population. There are two

general approaches to audit sampling: nonstatistical (targeted reviews) and statistical

(probability). In selecting the approach, the audit team considers variables such as the

audit budget, resources and time allocated, limitations and availability of data, and the

size of sampling population.

The QA reviewer is not required to be an expert on sampling. However, the reviewer is

responsible for performing a thorough review to ensure that Yellow Book requirements

are met. Some audit teams use sampling plans to document audit sampling procedures.

Therefore, QA reviewers may encounter this document during the QA review.

Good practices for the QA reviewer include the following:

• Verify that the sampling procedures or plan, methodology, results and

conclusions derived from audit sampling are reported accurately and supported

by adequate audit documentation.

• Determine if the sampling procedures or plan was approved according to the

agency’s policies and procedures, and whether the team followed the steps in

the approved sampling procedures or plan.

• Determine whether the audit team obtained approval or consulted with a

statistician or specialist to develop the sampling plan, if applicable.

• Determine whether the statistician or specialist has appropriate qualifications, if

applicable.

• Consult with the audit team, statistician, or specialist if there is difficulty

understanding audit sampling documentation.

The guidance in this white paper is not prescriptive; each QA reviewer should consider

the agency’s unique policies and procedures and use professional judgment in assessing

the agency’s implementation and compliance with professional standards. In addition,

this white paper should not be considered a replacement or supplement to generally

accepted government auditing standards, and it should not be considered as a basis for

an external peer review result.

Good Practices for Quality Assurance Reviewers: Audit Sampling Planning, Documentation, and Reporting

June 2021

In October 2016, representatives from various Federal Offices of Inspector General (OIG)

formed the Quality Assurance Working Group (QAWG) to enhance and improve the quality

assurance (QA) review processes used by the Federal OIG community. In January 2019, the

Council of the Inspectors General on Integrity and Efficiency (CIGIE) formally recognized

QAWG as part of the Federal Audit Executive Council (FAEC).

One of QAWG’s goals is to

identify and document good practices to help the OIG community improve their QA functions.

To implement this goal, QAWG, through FAEC, sent a survey in July 2019 to senior OIG audit

leadership and managers to identify key areas of concern about the application or interpretation

of performance audit standards. QAWG formed task teams to develop separate white papers that

address the top five identified areas of concern.

Purpose

This white paper provides good practices for performing a QA review of audit evidence obtained

from sampling, as well as good practices for documenting audit sampling. The guidance in this

white paper is not prescriptive; each QA reviewer should consider the agency’s unique and

procedures and use professional judgment in assessing the agency’s implementation and

compliance with professional standards. In addition, this white paper should not be considered a

replacement or supplement to generally accepted government auditing standards, and it should

not be considered as a basis for an external peer review result.

Background

When deciding whether to employ audit sampling techniques, the audit team considers the

specific audit objective(s) to be achieved and whether sampling procedures or a combination of

procedures to be applied will assist in efficiently achieving that objective(s). In planning and

conducting audits, it may not be effective, efficient, or practical to examine all of the available

data or entire population to achieve an audit objective. Instead, it may be useful to perform audit

sampling to draw valid conclusions and generalizations about a population. Audit sampling is the

selection and evaluation of less than 100 percent of the population, which the auditor expects to

be representative of the population and will provide a reasonable basis for conclusions about the

population. The audit team selects the appropriate sampling approach or method to evaluate the

population.

There are two general approaches to audit sampling: statistical (probability) and nonstatistical

(targeted reviews). Auditors may use either approach to evaluate the population. Both

approaches require that the auditor use professional judgment in planning, performing, and

FAEC is a subgroup established by CIGIE to discuss and coordinate issues affecting the Federal audit

community, with special emphasis on audit policy and operations of common interest to members.

The survey results identified the top concerns of OIG senior leadership and management where professional

standards were not being consistently interpreted: (1) audit risk, (2) data reliability, (3) sampling,

(4) supervisory review, and (5) quality assurance. Another key concern identified was related to internal

controls, which is being addressed by CIGIE’s separate internal controls working group.

INTRODUCTION

Good Practices for Quality Assurance Reviewers: Audit Sampling Planning, Documentation, and Reporting

June 2021

evaluating a sample and in relating the audit evidence produced by the sample to other audit

evidence when forming a conclusion about the population.

In selecting the approach, the audit

team considers other variables, such as the audit budget, resources and time allocated, limitations

and availability of data, and the size of sampling population.

Statistical Sampling

(Probability)

Statistical sampling, also referred to as probability sampling,

involves the use of techniques from which mathematically

constructed conclusions regarding the population can be drawn.

Statistical sample results are objective, defensible, and can be

projected to the population. See Appendix B: Commonly Used

Statistical Sampling Methods.

Nonstatistical Sampling

(Targeted Review)

Nonstatistical sampling, also referred to as targeted reviews, uses

non-probability methods such as judgment or experience to select

the sample. Recommendations are limited to the exceptions

derived from the sample. The results cannot be projected to the

population and should be reported with appropriate qualifying

language.

There are a number of reasons why an auditor may choose a nonstatistical sample instead of a

statistical sample, such as when the universe cannot be verified, the population is not easily

retrieved or available, or key data is missing. In certain situations, the audit team may determine

that sampling is not appropriate and decide to review 100 percent of the population. This is

commonly referred to as a census.

Criteria

The Yellow Book provides a framework for performing high-quality audit work with

competence, integrity, objectivity, and independence. According to the Yellow Book, auditors

must obtain sufficient, appropriate evidence to provide a reasonable basis for addressing the

audit objectives and supporting their findings and conclusions.

The degree of assurance

associated with a performance audit is strongly associated with the appropriateness of evidence

in relation to the audit objectives. The audit objectives might focus on verifying specific

quantitative results presented by the audited entity. In these situations, the audit procedures

would likely focus on obtaining evidence about the accuracy of the specific amounts in question.

This work may include the use of statistical sampling.

When a representative sample is needed,

statistical sampling generally results in stronger evidence than nonstatistical techniques. When a

representative sample is not needed, a targeted selection may be effective if the auditors have

isolated risk factors or other criteria to target the selection.

In reporting audit methodology, auditors should explain how the completed audit work supports

The use of sampling in financial audits differs somewhat from its use in performance audits. For more

information on sampling in financial audits, see the Financial Audit Manual published by GAO and CIGIE.

GAO, Government Auditing Standards (GAS), 8.90.

GAS, 8.103 and 8.103a.

GAS, 8.107.

Good Practices for Quality Assurance Reviewers: Audit Sampling Planning, Documentation, and Reporting

June 2021

the audit objectives, including the evidence-gathering and evidence-analysis techniques, in

sufficient detail to allow knowledgeable users of their reports to understand how the auditors

addressed the audit objectives. Auditors should identify the significant assumptions made in

conducting the audit; describe comparative techniques applied; describe the criteria used; and,

when the results of sample testing significantly support the auditors’ findings, conclusions, or

recommendations, describe the sample design and state why the design was chosen, including

whether the results can be projected to the intended population.

In addition to the Yellow Book requirements, auditors should also follow their agency’s internal

policies and procedures for addressing audit evidence and reporting audit methodology,

including specific requirements involving statistical sampling.

Audit sampling can be used to collect and analyze evidence during the audit. Deciding whether

to use audit sampling is a critical component to audit planning. It is considered early to ensure

that sampling procedures, or a combination of procedures, will assist in efficiently achieving

audit objectives. Sampling planning may include defining the objective; determining the type of

testing to be performed; and determining the population, statistical methodologies and

approaches, sample size, parameters, resources, data collection, and plans for analyzing and

interpreting the results. Sampling approaches and methodologies should be carefully considered

to ensure that results and conclusions are accurate.

Audit teams may also consider using a specialist, such as a statistician, early in the audit. The use

of specialists can enhance the credibility and quality of an audit and minimize the resources and

time needed to accomplish the objectives. A specialist can assist in acquiring, developing, and

framing a population; designing a statistical sampling plan; and performing other statistical

methods. This assistance can also include obtaining databases from outside sources, converting

data files to usable formats, identifying subpopulations, comparing counts for completeness, and

verifying totals for accuracy.

Audit documentation is an essential element of audit quality.

Auditors should prepare audit

documentation in sufficient detail to enable an experienced auditor with no previous connection

to the audit to understand the following from the audit documentation:

• the nature, timing, extent, and results of audit procedures performed

• the evidence obtained and its source

GAS, 9.14.

GAS, 8.137.

SAMPLING PLANNING

SAMPLING DOCUMENTATION

Good Practices for Quality Assurance Reviewers: Audit Sampling Planning, Documentation, and Reporting

June 2021

• the conclusions reached, including evidence that supports the auditors’ significant

judgments and conclusions

Auditors document sampling procedures in the work papers. The documentation may include the

following information:

• the audit objective

• the sample objectives or purpose

• the criteria applied and why appropriate

• the population, sampling unit, and sampling frame

• the sampling approach

• the selection methodology

• the sampling parameters, such as degree of reliability and confidence range (statistical

samples only)

• the sample size

• treatment of missed or omitted samples

• the data collection technique to be used

• a description of results

If audit sampling is used, it should be included in the audit team’s overall assessment of the

collective evidence used to support findings and conclusions.

Therefore, it is important that the

sampling procedures are thoroughly documented in the working papers. Some auditors may use

or are required to use a sampling plan or design to document sampling procedures. See Exhibit A

for an example.

Use of Specialist and Statisticians

Some audits may necessitate the use of specialized techniques or methods that call for the skills

of specialists.

Specialists and statisticians can perform various tasks during the sampling

process. For example, they can assist the audit team when more advance sampling

methodologies are involved, prepare or approve sampling plans and designs, and approve

sampling testing. In some cases, the statistician may perform or review the actual sampling

analysis to reach conclusions and projections.

When the results of sample testing significantly support the auditors’ findings, conclusions, or

recommendations, the report should describe the sample design and state why the design was

chosen, including whether the results can be projected to the intended population.

GAS, 8.132.

GAS, 8.108.

GAS, 4.13.

GAS, 9.14.

SAMPLING REPORTING

Good Practices for Quality Assurance Reviewers: Audit Sampling Planning, Documentation, and Reporting

June 2021

Good practices for reporting sampling results include the following:

• describing whether a statistical or non-statistical sample was used

• explaining why a sampling methodology was chosen

• describing the sample size tested in relation to the total population

• stating whether the results can be projected to the entire population

When using nonstatistical sampling, audit teams should indicate how the sample was selected

and include a statement that results from these samples cannot be used to make inferences about

the population.

Although this White Paper focuses on sampling, there are situations when auditors may choose a

census rather than sampling to achieve the audit objectives. A census is a test of 100 percent of

the population, and therefore does not require a projection. Conducting a census may be

appropriate when 100 percent reviews are possible due to the nature of the test, availability of

reliable data, when population sizes are small, or when no sampling uncertainty can be tolerated.

Similar to sampling procedures, the audit team documents the source and scope of the

population, accuracy of the data, how they tested the population data, results of their testing, and

the basis for conclusions reached.

QA Reviewer Independence

The QA reviewer should be independent of the audit team and should apply objectivity,

experience, analytical ability and knowledge of the Yellow Book to the task. Although

experience with sampling is desirable, it is not necessary for the QA reviewer to be an expert in

statistics or sampling.

Review of Report

The QA reviewer determines if the report contains the required information and explanations

concerning sampling. As a good practice, the QA reviewer should first gain an understanding of

the audit by reading the entire report, paying close attention to the audit objectives of the review

and whether sampling supports the audit’s objectives, results, and conclusions. It is important

that the sampling results are consistently interpreted, documented, and accurately reported. The

QA reviewer confirms that sampling information in the report is supported by adequate audit

documentation. The sampling results included in the report should tie back to the sampling

procedures and conclusions documented in the working papers.

NON-SAMPLING TECHNIQUE (CENSUS)

QUALITY ASSURANCE REVIEWER

RESPONBILITIES

Good Practices for Quality Assurance Reviewers: Audit Sampling Planning, Documentation, and Reporting

June 2021

The report is required to explain if the results of sample testing significantly support the findings,

conclusions, or recommendations. It should also describe the sample design and state whether

the results can be projected to the intended population. It may be necessary for auditors to

perform multiple samplings to achieve the audit objectives, and audit reports should therefore

provide this information in each instance of sample testing that significantly supports the

findings, conclusions, or recommendations. See Appendix C for examples of audit reports that

describe audit sampling.

Review of Audit Documentation

The audit documentation should provide the following types of information about sampling

procedures: audit objectives; sample objectives or purpose, criteria applied and why appropriate;

population, sampling unit, and sampling frame; sampling approach; selection methodology;

sampling parameters, such as degree of reliability and confidence range (statistical samples

only); sample size; treatment of missed or omitted samples; data collection technique to be used;

and a description of results. It is not necessary for the QA reviewer to determine the validity of

the sampling procedures. Rather, the task is to determine whether the audit team followed

sampling procedures that were approved according to the reviewed agency’s policies and

procedures. This may be found in the use of standardized audit steps and sampling procedures

provided by the agency for use in certain situations, supervisory approval, approval by a person

experienced in sampling techniques, or approval by a specialist such as a statistician. The QA

reviewer should also determine whether the audit team consulted with a statistician or specialist

to develop the sampling procedures.

Review of Specialists Qualifications

The QA reviewer may encounter instances where a specialist or statistician developed the

sampling procedures, performed the analysis, and/or developed conclusions. This is more likely

when complex sampling designs or techniques are used. The audit team should determine that

specialists assisting the audit team are qualified and competent in their areas of specialization.

The QA reviewer will typically find what is required to document qualifications in an agency’s

policies and procedures. The audit team should document the professional certification, license,

or other recognition of the competence of the specialist, as appropriate. For a comprehensive list

of information to document, see GAS, 4.15. The QA reviewer confirms that the specialist or

statistician had the appropriate qualifications, that the sampling procedures were approved

according the reviewed agency’s policies and procedures (this would typically be someone with

the appropriate qualifications such as a specialist or statistician), and that the work performed by

the specialist underwent a quality review by another specialist or statistician.

Review of Census

The QA reviewer may also encounter the results of a large and complex census. The QA

reviewer reviews these results by following the same principals for reviewing audit samples. In

GAS, 4.12.

Good Practices for Quality Assurance Reviewers: Audit Sampling Planning, Documentation, and Reporting

June 2021

all casesstatistical sample, nonstatistical sample, and censusan experienced QA reviewer

should avoid the temptation to recreate the entire analysis. Rather, the reviewer should focus on

determining if the language in the report ties back to the audit documentation, that the audit team

followed approved procedures, that complex procedures were designed and/or performed by

someone with appropriate qualifications, and that the work performed was appropriately

reviewed and reported.

If the QA reviewer has difficulty understanding the audit sampling or census documentation or

reporting, it is a good practice to consult with the audit team, specialists, or statisticians to

understand the nature, scope, and extent of the work performed, and that the approved

procedures were appropriately followed, reviewed, and reported.

Please see Appendix D for available training if QA reviewers want to learn more about sampling.

Good Practices for Quality Assurance Reviewers: Audit Sampling Planning, Documentation, and Reporting

June 2021

Audit/Project Number: _____________________________________________

Audit Objective: __________________________________________________

Prepared by: ______________________________________________________

Approved by: _____________________________________________________

Sampling Plan Purpose: Describe the sampling plan in accordance with the Yellow Book and

OIG policies and procedures. The sampling plan explains why statistical sampling is needed to

meet the audit objectives.

Population (Universe): Identify the audit population and specify its size, arithmetic mean, and

standard deviation. Be specific about what comprises your population.

Sampling Frame (Scope): The database (if applicable), other collection of data, timeframe, or

dollar amount, containing the totality of the sampling units from which the sample will be

selected.

Sampling Unit: The unit of analysis, which is any of the designated elements that comprise

the population of interest.

Sample Design (Methodology): The most common designs used are simple random,

stratified, cluster, and multistage sampling.

Measurement Characteristics (Criteria): Describe the sample’s criteria and characteristics.

Sample Size: The sampling plan should explain the sample size that must be sufficient to meet

the audit objectives. Specify the parameters used to determine the sample size for statistical

samplesfor example, confidence level and margin of error.

Source of Random Numbers and Sample Selection: The sampling plan should describe the

source of the random numbers used to select sample itemsfor example, Excel RAND

formula.

Estimation Methodology: Describe the estimates to be reported, the rationale for using the

estimates, and how the estimates will be calculated. Specify the parameters used to determine

the estimates for statistical samplesfor example, confidence interval, and confidence level.

Treatment of Missed or Omitted Samples: Describe how missing or omitted sampling units

will be treated.

Description of How Results Will Be Reported: Describe how the results will be reported.

The audit team should be able to envision how the results of the sample will be used and

reported and should know the objectives before the sample is selected.

APPENDIX A: SAMPLING PLAN TEMPLATE

Good Practices for Quality Assurance Reviewers: Audit Sampling Planning, Documentation, and Reporting

June 2021

Statistical Sampling

Statistical sampling helps the auditor design an efficient sample, measure the sufficiency of the

evidential matter obtained, and evaluate the sample results. Results of the sample tested can be

projected to population with measurable precision. Sample size can be based on a variable (for

example, dollar unit) based on error rate and precision goals. It can also be based on attributes

(for example, discovery, one-step, two-step, and rate estimation) based on error rate and risk.

Statistical sampling has the following benefits:

• It employs probability theory; each item has a known probability of being selected.

• It estimates the sample size objectively.

• Sample results are objective and defensible.

Cluster Sampling

Cluster sampling is a procedure in which groups of items, rather than individual items, are

selected for testing. Each sampling unit is called a cluster. Cluster sampling tends to be a small-

scale similar version of the entire population, such as business units, geographic locations,

categories of school (primary, secondary, etc.), or asset categories (property, equipment,

vehicles, etc.). Population is divided into clusters, and then groups of clusters are selected

randomly for sampling or examined entirely.

Estimation Sampling

The goal of this type of sampling is to estimate the actual noncompliance rate in a population

with a level of precision specified by the audit team. Methodologies used under this estimation

include simple random sampling, stratified random sampling, and systematic random sampling.

For estimation sampling, the sample size should be discussed. It can be used for attribute and

variable sampling and allows the auditor to project the error rate in the sample to the universe.

• Sample random sampling. This is the simplest method of drawing a statistical sample;

the design is the basis of all the other sampling designs. Simple random sampling uses

techniques to ensure that every item of the population has an equal chance of selection.

Selection can be performed and documented using software that generates random

numbers such as CaseWare IDEA, SAS (Statistical Analysis System), or Microsoft

Excel.

• Stratified random sampling. This method divides a population into sub-populations,

each of which is a group of sampling units with similar characteristics. This includes

items such as monetary value, region, size, and of type of organization. Generally, a

simple random sample is selected for each part (stratum).

APPENDIX B: COMMONLY USED STATISTICAL

SAMPLING METHOD

Good Practices for Quality Assurance Reviewers: Audit Sampling Planning, Documentation, and Reporting

June 2021

An estimate is determined separately for each stratum, and these are combined to form an

estimate for the entire population. When presenting results from a stratified sample,

estimates should be calculated incorporating sampling weights used.

• Multistage sampling. This method divides large populations into stages to make the

sampling process more practical. It requires two or more stages of sampling to achieve

greater efficiency. Various sampling methods may be combined later at different stages.

Systematic Sampling

Systematic sampling involves selecting items from the population at a given interval after

establishing a random starting placefor example, the selection of every ‘’nth” item following a

random start. The random start is essential to ensure that each item in the population has an equal

chance to be included in the sample. The population and required sample size should be

estimated in order to determine the interval necessary. A random starting point between one and

the interval is then obtained as a starting point, and the interval is added to it until the desired

sample size is reached.

Good Practices for Quality Assurance Reviewers: Audit Sampling Planning, Documentation, and Reporting

June 2021

Example 1: The United States Department of Health and Human Services, Office of the

Inspector General, Audit Report, A-07-16-03209, March 1, 2017

https://www.oversight.gov/sites/default/files/oig-reports/71603209.pdf

Example 2: The United States Government Accountability Office, GAO Report,

GAO-18-419, May 30, 2018

https://www.gao.gov/assets/700/692155.pdf

Example 3: The United States Social Security Administration, Office of the Inspector General,

Audit Report, A-02-14-31417, July 30, 2020

https://www.oversight.gov/sites/default/files/oig-reports/A-02-14-31417.pdf

Example 4: The United States Agency for International Development, Office of the Inspector

General, Audit Report, 9-000-19-006-P, September 25, 2019

https://www.oversight.gov/sites/default/files/oig-reports/9-000-19-006-P.pdf

APPENDIX C: EXAMPLES OF AUDIT REPORTS

THAT DESCRIBE AUDIT SAMPLING

Good Practices for Quality Assurance Reviewers: Audit Sampling Planning, Documentation, and Reporting

June 2021

USDA Graduate School

Practical Statistical Sampling for Auditors

https://www.graduateschool.edu/content/gati

Institute of Internal Auditors

Data Sampling

https://ondemand.theiia.org/learn/course/external/view/elearning/157/data-sampling

Data Analysis and Sampling

https://na.theiia.org/training/courses/Pages/Data-Analysis-and-Sampling.aspx

Data Analysis for Internal Auditors

https://na.theiia.org/training/courses/Pages/Data-Analysis-for-Internal-Auditors.aspx

Management Concepts

Data Collection Techniques

https://www.managementconcepts.com/course/id/4610

Data Analysis and Modeling Techniques

https://www.managementconcepts.com/course/id/4615

Analytics Boot Camp: Core Analytics

https://www.managementconcepts.com/course/id/4607

APPENDIX D: AVAILABLE SAMPLING TRAINING

Good Practices for Quality Assurance Reviewers: Audit Sampling Planning, Documentation, and Reporting

June 2021

Audit: Either a financial audit or performance audit conducted in accordance with generally

accepted government auditing standards (GAGAS). (GAS, 1.27b)

Audit Organization: A government audit entity or a public accounting firm or other audit entity

that conducts GAGAS engagements. (GAS, 1.27c) Audit organization and Office of Inspector

General (OIG)—that either with or without an audit function performs GAGAS engagements—

are used interchangeably in this white paper.

Audit Risk: The possibility that the auditors’ findings, conclusions, recommendations, or

assurance may be improper or incomplete. The assessment of audit risk involves both qualitative

and quantitative considerations. (GAS, 8.16)

Audit Sampling: The selection and evaluation of less than 100 percent of the population, which

the auditor expects to be representative of the population and will provide a reasonable basis for

conclusions about the population.

Council of Inspector General on Integrity and Efficiency (CIGIE): An independent entity

statutorily established within the executive branch by The Inspector General Reform Act of

2008, P.L. 110-409, to address integrity, economy and effectiveness issues that transcend

individual Government agencies; and increase the professionalism and effectiveness of personnel

by developing policies, standards, and approaches to aid in the establishment of a well-trained

and highly skilled workforce in the offices of the Inspectors General. https://ignet.gov

Federal Audit Executive Council (FAEC): A subgroup, established by CIGIE, to discuss and

coordinate issues affecting the Federal audit community with special emphasis on audit policy

and operations of common interest to FAEC members. https://ignet.gov/content/federal-audit-

executive-council

GAO: Government Accountability Office. Known as "the investigative arm of Congress" and

"the congressional watchdog," GAO supports Congress in meeting its constitutional

responsibilities and helps improve the performance and accountability of the Federal government

for the benefit of the American people.

Nonstatistical Sampling: Also referred to as targeted reviews, this approach uses non-

probability methods to select the sample, such as judgment and experience. Recommendations

are limited to the exceptions derived from the sample. The results cannot be projected to the

population and should be reported with appropriate qualifying language.

Performance audits: Engagements that provide objective analysis, findings, and conclusions to

assist management and those charged with governance and oversight to, among other things,

improve program performance and operations, reduce costs, facilitate decision making by parties

with responsibility to oversee or initiate corrective action, and contribute to public

accountability. In a performance audit, the auditors measure or evaluate the subject matter of the

APPENDIX E: GLOSSARY

Good Practices for Quality Assurance Reviewers: Audit Sampling Planning, Documentation, and Reporting

June 2021

audit and present the resulting information as part of, or accompanying, the audit report. (GAS,

1.21 and 8.14)

Quality Assurance (QA): An ongoing consideration and evaluation of the audit organization’s

system of quality control, including inspection of engagement documentation and reports for a

selection of completed engagements to provide management with reasonable assurance that (1)

the policies and procedures related to the system of quality control are suitably designed and

operating effectively in practice and (2) auditors have followed professional standards and

applicable legal and regulatory requirements. GAGAS also refers to this process as “monitoring

of quality.” (GAS, 5.47)

Quality Assurance (QA) Review: The performance, documentation, and communication of

monitoring procedures and results that enable the audit organization to assess compliance with

professional standards and quality control policies and procedures for completed GAGAS

engagements. Reviews of the work by engagement team members prior to the date of the report

are not monitoring procedures. (GAS, 5.43, 5.44, 5.47, 5.53, 5.59)

Quality Assurance (QA) Reviewer: An individual who performs monitoring procedures and

assesses the audit organization’s compliance with professional standards and quality control

policies and procedures for GAGAS engagements. The individual should have sufficient

expertise and authority with the audit organization and, if possible, does not have responsibility

for the specific activity being reviewed. (GAS, 5.43, 5.48)

Quality Assurance Working Group (QAWG): A group formed by representatives from

various Federal Offices of Inspector General in October 2016 to enhance and improve the quality

assurance review processes within the Federal Inspector General community and that formally

became a subgroup under the CIGIE FAEC in January 2019.

https://ignet.gov/sites/default/files/files/QAWG-Charter.pdf

Quality Control: The OIG’s leadership and policies and procedures designed to provide the

audit organization with reasonable assurance that the organization and its personnel comply with

professional standards and applicable legal and regulatory requirements. The nature, extent, and

formality of an audit organization’s quality control system will vary based on the audit

organization’s circumstances, such as size, number of offices and geographic dispersion,

knowledge and experience of its personnel, nature and complexity of its engagement work, and

cost-benefit considerations. (GAS, 5.02, 5.03)

Statistical Sampling: Also referred to as probability sampling, this approach uses techniques

from which mathematically constructed conclusions regarding the population can be drawn.

Statistical sample results are objective, defensible, and can be projected to the population. See

Appendix B: Commonly Used Statistical Sampling Methods.

U.S. Government Accountability Office’s (GAO) Government Auditing Standards, 2018

Revision (April 2021), GAO-21-368G: This publication (known as the Yellow Book or GAS)

prescribes professional standards that provide a framework for auditors to perform high-quality

audit work with competence, integrity, objectivity, and independence to help improve

government operations and services. These professional standards are often referred to as

Good Practices for Quality Assurance Reviewers: Audit Sampling Planning, Documentation, and Reporting

June 2021

generally accepted government auditing standards (GAGAS).

https://www.gao.gov/assets/gao-21-368g.pdf

In April 2021, GAO made technical updates to the 2018 revision of Government Auditing Standards. These

technical updates to the 2018 revision of Government Auditing Standards were effective upon issuance. For

additional information, please see GAO-21-368G, pp. i-ii.

Good Practices for Quality Assurance Reviewers: Audit Sampling Planning, Documentation, and Reporting

June 2021

Team Member Office of Inspector General

Scott A. Spaulding (Co-lead) Department of Justice

Sylvester Tang (Co-lead) U.S. Agency for International Development

Brittany Banks Corporation for National & Community Service

Jerri Dorsey-Hall Environmental Protection Agency

Ed Gold (Editor) Amtrak

APPENDIX F: LIST OF CONTRIBUTORS