inspections.md

Inspection configuration

Each consignment is inspected according to the rules under the inspection configuration key. Optionally, inspection of some consignments can be skipped using a release programs.

Skipping inspections

Before an inspection occurs, a release programs, specified under the key release_programs, can be used to skip an inspection of a particular consignment.

Several release programs including Dynamic Skip Lot Program and Cut Flower Release Program are available, but only one program can be specified at a time.

By default, first release program under release_programs is used. When name of a release program is provided as release_program under inspection, a release program with the matching name is used. The names are the same as the keys under release_programs (see below). For example, the Cut Flower Release Program (CFRP) would be explicitly enabled using:

inspection:
  release_program: cfrp
release_programs:
  cfrp:
    schedule:
      file_name: schedule.csv

If an empty value (None in Python, null in JSON) is provided as release_program, no release program is activated. Additionally, any name which is not present in the release_programs will avoid release program activation. For scenarios, the configuration may use none as the name to avoid release program activation, while still using a text (string) value rather than special empty value.

Fixed Skip Lot Program

List of compliance levels and their associated ratios of inspected consignments:

release_programs:
  fixed_skip_lot:
    levels:
      - name: 1
        sampling_fraction: 1
      - name: 2
        sampling_fraction: 0.5
      - name: 3
        sampling_fraction: 0

List of consignment properties (attributes) which determine into which compliance level a consignment belongs to:

    track:
      - origin
      - commodity

List which assigns values of tracked consignment properties to compliance levels:

    consignment_records:
      - origin: Netherlands
        commodity: Hyacinthus
        compliance_level: 2
      - origin: Mexico
        commodity: Gerbera
        compliance_level: 3

Default compliance level when a consignment does not have a compliance level defined:

    default_level: 1

Dynamic Skip Lot Program

In the dynamic skip lot program, consignment dynamically fall to compliance levels based on inspection results. The proportion of inspected consignments for the same combination of consignment properties dynamically changes based on previous simulation steps. The program is identified by the key dynamic_skip_lot. An optional name can be added and will appear in the output.

release_programs:
  dynamic_skip_lot:
    name: Dynamic Skip Lot

Inspection results are tracked for consignments grouped based on tracked consignment one or more properties specified as a list with track.

track:
  - origin
  - commodity

Each group is assigned a level that determines the frequency of inspections. Each compliance level has an associated fraction of consignments to be inspected (sampling_fraction). The name key for a level is optional, sampling_fraction is required. If names are provided, they need to be unique.

levels:
  - name: Compliance Level 1
    sampling_fraction: 1
  - name: Compliance Level 2
    sampling_fraction: 0.5
  - name: Compliance Level 3
    sampling_fraction: 0.25
  - name: Compliance Level 4
    sampling_fraction: 0.1

A random number is generated for each consignment to determine whether to inspect. The number is compared to the sampling_fraction of the current compliance level to determine whether or not to inspect the particular consignment. On average, a sampling fraction of consignments are marked for inspection.

The order of the levels is important, as groups move through the levels from the first one in the list to the last one in the list. All groups start at the first level or a custom start level specified with start_level, which can refer to the level either by name or by a one-based index (the default start level is 1).

start_level: Compliance Level 1

To advance to a higher level, a group must reach a certain number of consecutive successful inspections, called the clearance number.

clearance_number: 10

The tracking for the clearance number is reset after a group moves up a level. If a consignment fails an inspection, the level for the group it belongs to is reset to the start level. The consignment group can then move up the levels in the standard manner. If the group is already at or below the start level, its current level is decreased by one level.

If monitoring_level is set, the group moves to the monitoring level instead of start level. Just like start_level, monitoring_level can be set either by name or by a one-based index. Start level and monitoring_level can be combined together in any way, for example:

start_level: Compliance Level 2
monitoring_level: Compliance Level 4

If the group's current level is already at or lower than the monitoring level, and a consignment fails the inspection, its current level is decreased by one level just like when start_level is used and monitoring_level is not set.

As an alternative to monitoring, a failed inspection will result in decreasing the compliance level for the group by one or any other number of levels specified by decrease_levels, for example:

decrease_levels: 2

The compliance level is never decreased bellow the first level defined in levels. decrease_levels can be specified as integer or boolean (true or false). If it is true, the decrease is done by one level.

The groups can quickly move back to their original level before failing inspection if quick reinstating of the original level is enabled with quick_reinstating.

quick_reinstating: true

While the clearance_number is used for reinstating by default, an additional lower clearance number for reinstating can be specified with quick_reinstate_clearance_number. If quick_reinstate_clearance_number is provided, quick reinstating is automatically enabled even if quick_reinstating is not provided.

quick_reinstate_clearance_number: 5

Naive Cut Flower Release Program

A prototype implementation of a simple theoretical release program modeled after the Cut Flower Release Program (CFRP) is included in the simulation as Naive Cut Flower Release Program (naive_cfrp). When activated, only one kind of flowers is inspected each day (a day is based on the date which is a attribute of the consignment). The program is applied to consignments which have flowers specified in flowers and which have less then max_boxes. The program parameters can be specified like this:

release_programs:
  naive_cfrp:
    flowers:
      - Hyacinthus
      - Gerbera
      - Rosa
      - Actinidia
    max_boxes: 10  # do not apply to consignments larger than

Cut Flower Release Program

Main article: Cut Flower Release Program

A full Cut Flower Release Program (cfrp) is included in the simulation and is driven primarily by a schedule which determines qualifying flowers (or generally, commodities) and Flower of the Day (FotD), a flower to inspect on a given day. A minimal configuration looks like this:

release_programs:
  cfrp:
    schedule:
      file_name: schedule.csv

Effectiveness

Contaminated items can be either always be detected during the inspection, or only a certain percentage of them can be detected based on the effectiveness. The effectiveness is a value between 0 and 1. The following will cause 1 out of 10 contaminated items on average to pass undetected.

inspection:
  effectiveness: 0.9

Inspection unit

The inspection unit can be determined by:

inspection:
  unit: items

The inspection unit can be either items for computing sample size based on number of items in the consignment or boxes for computing sample size based on number of boxes in the consignment.

Partial box inspections

The proportion of items within a box to be inspected can be determined by:

inspection:
  within_box_proportion: 1

The value of within_box_proportion can be set to any value greater than 0 and less than or equal to 1. By default, within_box_proportion = 1 meaning all items within a box can be inspected. If within_box_proportion < 1, only the first n = within_box_proportion * items_per_box items in each box will be inspected.

Tolerance level

The simulation provides a count of the number of missed consignments (slippage) with contamination rates below the specified tolerance_level, which can be any value between 0 and 1. The resulting count can be used to adjust the number of missed contaminated consignments when evaluating inspection efficacy. For example, if an inspection strategy fails to detect 100 out of 500 contaminated consignments (20% slippage), but 80 of the missed consignments had a contamination_rate < tolerance_level, you may choose to adjust the inspection slippage rate to only include the 20 consignments with contamination rates above the tolerance level (decreases to 4% slippage). Note that the tolerance level affects the reported statistics, but does not change the mechanics of the simulated inspections.

The contamination tolerance level can be determined by:

inspection:
  tolerance_level: 0

Sample strategy

The sample strategy defines the method used to compute the number of units to inspect. The sample strategy can be determined by:

inspection:
  sample_strategy: proportion

The possible sample strategies include proportion for sampling a specified proportion of units, hypergeometric for sampling to detect a specified contamination level at a specified confidence level using the hypergeometric distribution, fixed_n for sampling a specified number of units, and all for sampling all units.

Proportion strategy

The settings for proportion are:

inspection:
  sample_strategy: proportion
  proportion:
    value: 0.02
    min_boxes: 1

The proportion value is set by proportion and the minimum number of boxes to be inspected is set by min_boxes. If unit = "items", the number of items to inspect is computed by proportion * num_items. Similarly, if unit = "boxes", the number of boxes to inspect is computed by proportion * num_boxes.

Hypergeometric strategy

The settings for hypergeometric are:

inspection:
  sample_strategy: hypergeometric
  hypergeometric:
    detection_level: 0.05
    confidence_level: 0.95
    min_boxes: 1

The minimum contamination level to be detected with sample is set by detection_level at the confidence level set by confidence_level. The minimum number of boxes to be inspected is set by min_boxes. The sample size is calculated using a hypergeometric distribution (sampling without replacement) as described in (Fosgate, 2009). The equation used to compute the sample size is:

$$n=(1-(alpha)^1/D*N)(N-(D*N-1/2))$$

where alpha is 1 - confidence_level, D is detection_level, and N is num_units.

Fixed n strategy

The settings for fixed_n are:

inspection:
  sample_strategy: fixed_n
  fixed_n: 10

The number of units to be inspected can be any integer set by fixed_n. If unit = "items", the sample size will be set to the minimum of two values: fixed_n and the maximum number of inspectable items based on within_box_proportion (max_items = within_box_proportion * items_per_box * num_boxes). If unit = "boxes", the sample size will be set to fixed_n if min_boxes <= fixed_n <= num_boxes. If fixed_n is less than the minimum number of boxes to inspect, number of boxes to inspect will be set to min_boxes. If fixed_n exceeds the number of boxes in the consignment, number of boxes to inspect will be set to num_boxes.

Selection strategy

While the sample strategy determines how many units to inspect, the selection strategy is used to determine which units to select for inspection. The unit selection strategy can be determined by:

inspection:
  selection_strategy: random

The possible selection strategies include random for selecting units to inspect using a uniform random distribution, convenience for selecting the first n units to inspect, or cluster for selecting boxes for partial inspection. The cluster selection strategy is valid only for the item inspection unit.

Random selection strategy

The random selection strategy selects units to inspect using a uniform random distribution. Each unit in the consignment has an equal probability of being selected. This strategy requires more effort from inspectors but provides a higher quality, more representative sample of the consignment. Random selection is more robust when contaminants may be distributed non-uniformly throughout the consignment.

Convenience selection strategy

The convenience selection strategy selects the first n units to inspect. This approach requires less effort as inspectors only examine the most accessible items (e.g., items at the front of boxes or top of the consignment). However, convenience sampling may introduce bias if contaminants are not uniformly distributed, potentially resulting in higher slippage rates compared to random selection.

Cluster selection strategy

The cluster selection strategy is used when inspection_unit="item" to divide the sample size into clusters that can be selected from boxes chosen either randomly (cluster_selection="random") or at a systematic interval (cluster_selection="interval"). This selection strategy cannot be used with the highest inspection unit (e.g., unit = "boxes").

The settings for cluster are:

inspection:
  within_box_proportion: 0.1
  cluster:
    cluster_selection: random
    interval: 3

The method for selecting the cluster units (boxes in this case) is set by cluster_selection. The possible values for cluster_selection are random for selecting the cluster units using a random uniform distribution and interval for selecting every nth cluster unit. If cluster_selection="interval", the interval size is set by interval. The proportion of items to inspect within each cluster unit is set by within_box_proportion.

A simple example using the following configuration:

consignment:
  items_per_box:
    default: 200
inspection:
  unit: item
  within_box_proportion: 0.25
  sample_strategy: hypergeometric
  selection_strategy: cluster
  cluster:
    cluster_selection: random

In this case, the sample size is calculated using the hypergeometric approach based on the total number of items in the consignment and the items to be inspected are selected using the cluster approach.

Let's say the computed sample size (n_units_to_inspect) is 200 items. The following steps are used to determine which items to inspect. First, the within_box_proportion value is used to determine how many clusters (i.e., boxes) are needed to get to the sample size. The number of items inspected per box is within_box_proportion * items_per_box (200 * 0.25 = 50), so only 50 items would be inspected per box. The number of boxes that need to be opened to get to the sample size is n_units_to_inspect / inspect_per_box (200 / 50 = 4), so 4 boxes need to be opened. Once the number of boxes needed is determined, those boxes are then selected from the consignment randomly since cluster_unit = "random" and the first 25% of items are inspected convenience style, i.e., first n or tailgate.

The cluster selection strategy could be useful when using the hypergeometric sample strategy with items as the inspection unit. An important assumption for the hypergeometric sample size to work effectively is that every unit has an equal chance of being selected. This is feasible when using boxes as the inspection unit, since boxes could be numbered and the inspector could be directed to select boxes based on randomly generated box numbers. When using items as the sample unit, however, ensuring every item has an equal chance of being selected is more difficult, as there are likely too many items to individually number and the items may be packaged in bunches and stacked within a box. The cluster selection method does not ensure each item has an equal chance of being selected, but it provides a sort of compromise to spread the sample out across the consignment while still limiting the number of boxes opened and items inspected.

End strategy

Each simulation automatically runs two options for determining when to end an inspection (end strategies). The two possible end strategies are to detection and to completion.

For the to detection end strategy, the inspection ends as soon as a contaminant is detected. For the to completion strategy, the inspection continues until the full sample has been inspected, regardless of contaminant detection.

The number of contaminated units detected for each end strategy is compared to quantify the proportion of contaminants reported when the inspection is ended at detection.

---

Next: Run simulations