AUDITING FUNCTION POINT COUNTS
The very term auditor and audit makes
many of us feel uncomfortable. Many industries have independent inspectors
and auditors. As function points become more widely used and become a
more important part of decision making, those using the function point
counts will want them independently reviewed and audited. Auditing is
the process by which a competent, independent person accumulates and evaluates
evidence about quantifiable information related to a specific entity for
the purpose of determining and reporting on the degree of correspondence
between the quantifiable information and established criteria.
Like other industries that have prescribed
guidelines (such as accounting), independent auditors can provide valuable
feedback on the actual function point count and the overall function point
counting process. An function point auditor should be independent, but
an auditor can be still be inside your company, perhaps part of your metrics
team, or they can be an independent third party.
To do an audit of any kind, there must be information in a verifiable form and some standard (hopefully IFPUG 4.1) by which the auditor can evaluate the information. The auditor may go to the business premises to examine records and obtain information about the reliability of the function point counts. On the other hand, there may be adequate information that can be sent to the function point auditor that can be reviewed off site.
Accumulating and Evaluating Evidence
Reviewing a function point count to insure
IFPUG 4.1 counting guidelines were followed would be considered a compliance
audit. The purpose of a compliance audit is to determine whether the function
point counts follow specific procedures and guidelines set down by the
IFPUG Counting Practices Committee. The results of a compliance audit
are generally reported to someone within the organizational unit being
audited rather than to a broad spectrum of users.
Evidence is defined as any information
used by the auditor to determine whether the function point count being
audited is in compliance with IFPUG guidelines. Evidence can take many
different forms, the function point count, system documentation, conversations
with developers and users, and interviews with individuals that conducted
the original count. The auditor gathers evidence to draw conclusions.
Of course the function point count itself
can be used as evidence, but using the function point count alone would
be severely inadequate. It is impossible to determine the accuracy of
a function point count without evaluating additional evidence.
If an auditor was given the task of auditing
a company with 500,000 function points it would be impossible to review
every count. The auditor may select only 20 or 30 applications to actually
audit. The actual sample size will very from auditor to auditor and audit
to audit. The decision of how many items to test must be made by the auditor
for each audit procedure. There are several factors that determine the
appropriate sample size in audits. The two most important ones are the
auditors' expectations of the number of errors and the effectiveness of
the clients internal function point counting procedures. The suggested
procedures at the end of this document can help in determining both of
Additionally, the evidence must be pertain
or be relevant to the audit. The auditor must be skilled at finding areas
to test or review further. For example, the auditor may determine during
conversations that their was some confusion about external inputs and
external interface files. In this case, the auditor would review the actual
system documentation and the function point count to insure that the all
the external input and external interface file were treated correctly.
Another example, would be that the function point counter had never counted
a GUI application. The auditor would review a series of screens and determine
if the original counter had correctly counted such items as radio buttons,
check boxes, and so on.
The evidence must be considered believable
or worthy of trust. If evidence is considered highly trusty worthy, it
is a great help in assisting the auditor with a function point audit.
On the other hand, if the evidence is in question such as incomplete documentation
(or old documentation) then the auditor would have to scrutinize these
areas of the count more closely. Additionally, the auditor should make
note in the final report of any evidence they requested and the client
was not able to provide.
All evidence should be evaluated based
upon valuation, completeness, classification, rating, mechanical accuracy,
and analytical analysis.
Valuation: The objective deals
with whether items included in the function point count should of been
included. Perhaps the original function point count included additional
transactions or files that should not of been included.
Completeness: The objective deals
with including all transactions and files in the final function point
count. It is important that the application team review the final function
point count to insure all transactions and files have been included. The
valuation and completeness objectives emphasize opposite audit concerns.
Valuation deals with potential overstatement and completeness with unrecorded
transactions and files.
involves determining whether all transactions and files have been correctly
classified. It is important to make sure the external input and external
interface file have been classified correctly for example.
Rating: This objective deals with
determining if the transactions and files were appropriately ranked as
low, average or high. To complete this objective a detail examination
of the data elements and files referenced.
Mechanical Accuracy: Testing the
mechanical accuracy involves rechecking a sample of the computations and
transfers of information from one document to the next. Rechecking of
computations consists of testing the original function point counters
arithmetical accuracy. This is most important if an automated tool was
not used while counting function points.
Analytical Analysis: This procedure
is another way that a function point count can be validated. For example,
the ratio of external inputs, external output, external inquiry, internal
logical file, and external interface file can be compared with other applications
meeting similar business needs. Also, the general system characteristics
can be reviewed and compared to similar applications. Analytical procedures
should be performed early in the audit so to help the auditor determine
areas that need to be more thoroughly investigated.
Before an audit or validation of a function point count can take place a procedure should be in place to evaluate a the count(s). A procedure, at a minimum, should cover all the areas mentioned above. The procedure does not have to be a rigid document that is followed, but a guideline to conduct the audit. At the end of this article is a 20 step procedure that should assist anyone with developing their own guidelines or with auditing a function point count.
The auditor must be a qualified to understand
the criteria used and competent to know the types and amount of evidence
to accumulate to reach the proper conclusion after the evidence has been
examined. The auditor also must have an independent mental attitude. It
does little good to have a competent person who is biased performing the
Independence cannot be absolute by any means, but it must be a goal. For example, even though an auditor may be paid a fee by a company they may still be sufficiently independent to conduct audits that can be relied upon by users. Auditors may not be sufficiently independent if they are also company employees.
Types of Reports
The final stage of the audit process
is the audit report - the communication of findings to the users. Reports
differ in nature, but in all cases they must inform readers of the compliance
with IFPUG Counting Guidelines. The auditor can have three types of reports.
Like in all audits, the most common report
should be the Standard Unqualified Audit Report. It is used in 90 percent
of all audits, and a function point audit should be no different. The
standard unqualified report is used when the following conditions are
Another type of audit report is "conditions
requiring a departure." There are two conditions requiring a departure
from a Standard Unqualified Audit.
Additionally, the auditor may create
an adverse opinion. An adverse opinion is used only when the auditor believes
the overall function point counts are so materially misstated or misleading
that they do no present fairly the functional size of the application
being counted. The auditor should be very specific on why they are making
A 20 Step Procedure for Auditing a Function
For more information:
to Share — to copy, distribute and transmit the work
|Keywords,Software Estimating, Benchmark, Software industry data, Software Productivity, function points, FPA, IFPUG, Initial Public Offerings, Merger, Acquisitions, Software Measurement,software metrics, CMM, benchmark studies, function point counting, outsourcing metrics, offshore, offshore measurement,venture capital,sizing software, software economics, economics of software, expert testimony, software litigation, IPO, software IPO, benchmark data, training,consulting,software productivity, industry data, productivity data, software estimating, process metrics|