mddc

Contents

mddc#

MDDC.MDDC.mddc(contin_table, method='monte_carlo', rep=10000, quantile=0.95, exclude_same_drug_class=True, col_specific_cutoff=True, separate=True, if_col_corr=False, corr_lim=0.8, coef=1.5, chunk_size=None, n_jobs=-1, seed=None)#

Modified Detecting Deviating Cells (MDDC) algorithm for adverse event signal identification.

This function implements the MDDC algorithm using either a Monte Carlo or Boxplot method for cutoff selection. The Monte Carlo or Boxplot method is used to estimate thresholds for identifying cells with high values of standardized Pearson residuals,

Parameters#

contin_tablepd.DataFrame or np.ndarray

A contingency table of shape (I, J) where rows represent adverse events and columns represent drugs or vaccines. If a DataFrame, it might have index and column names corresponding to the adverse events and drugs/vaccines.

methodstr, optional, default=”monte_carlo”

Method for cutoff selection. Can be either “monte_carlo” or “boxplot”.

repint, optional, default=10000

Number of Monte Carlo replications used for estimating thresholds. Utilized in Step 2 of the MDDC algorithm. Only used if method is “monte_carlo”.

quantilefloat, optional, default=0.95

The quantile of the null distribution obtained via Monte Carlo method to use as a threshold for identify cells with high value of the standardized Pearson residuals. Used in Step 2 of the MDDC algorithm. Only used if method is “monte_carlo”.

exclude_same_drug_classbool, optional, default=True

If True, excludes other drugs/vaccines in the same class when constructing 2x2 tables for Fisher’s exact test. Only used if method is “monte_carlo”.

col_specific_cutoffbool, optional, default=True

Apply Monte Carlo/Boxplot method to the standardized Pearson residuals of the entire table, or within each drug/vaccine column. If True, applies the Monte Carlo/Boxplot method to residuals within each drug/vaccine column. If False, applies it to the entire table. Utilized in Step 2 of the algorithm.

separatebool, optional, default=True

Whether to separate the standardized Pearson residuals for the zero cells and non zero cells and apply Monte Carlo/Boxplot method separately or together. If True, separates zero and non-zero cells for cutoff application. If False, applies the cutoff method to all cells together. Utilized in Step 2 of MDDC algorithm.

if_col_corrbool, optional, default=False

Whether to use column (drug/vaccine) correlation or row (adverse event) correlation If True, uses drug/vaccine correlation instead of adverse event correlation. Utilized in Step 3 of the MDDC algorithm.

corr_limfloat, optional, default=0.8

Correlation threshold used to select connected adverse events. Utilized in Step 3 of MDDC algorithm.

coefint, float, list, numpy.ndarray, default = 1.5

Used only when method = boxplot. A numeric value or a list of numeric values. If a single numeric value is provided, it will be applied uniformly across all columns of the contingency table. If a list is provided, its length must match the number of columns in the contingency table, and each value will be used as the coefficient for the corresponding column.

chunk_sizeint, optional, default=None

Useful in scenarios when the dimensions of the contingency table is large as well as the number of Monte Carlo replications. In such scenario the Monte Carlo samples need to be generated sequentially such that the memory footprint is manageable (or rather the generated samples fit into the RAM).

n_jobsint, optional, default=-1

n_jobs specifies the maximum number of concurrently running workers. If 1 is given, no joblib parallelism is used at all, which is useful for debugging. For more information on joblib n_jobs refer to - https://joblib.readthedocs.io/en/latest/generated/joblib.Parallel.html.

seedint or None, optional, default=None

Random seed for reproducibility.

Returns#

resultnamedtuple
  • If method is “monte_carlo” returns MDDCMonteCarloResult:
    • pvalnumpy.ndarray, pd.DataFrame

      p-values for each cell in the step 2 of the algorithm, calculated using the Monte Carlo method (Algorithm 3 of Liu et al.(2024)).

    • pval_fishernumpy.ndarray, pd.DataFrame

      p-values for each cell in the step 2 of the algorithm, calculated using the Monte Carlo method for cells with count greater than five, and Fisher’s exact test for cells with count less than or equal to five.

    • signalnumpy.ndarray, pd.DataFrame

      Matrix indicating significant signals with count greater than five and identified in the step 2 by the Monte Carlo method. 1 indicates a signal, 0 indicates non-signal.

    • fisher_signalnumpy.ndarray, pd.DataFrame

      Matrix indicating signals with a count less than or equal to five and identified by Fisher’s exact test. 1 indicates a signal, 0 indicates non-signal.

    • corr_signal_pvalnumpy.ndarray, pd.DataFrame

      p-values for each cell in the contingency table in the step 5, when the \(r_{ij}\) (residual) values are mapped back to the standard normal distribution.

    • corr_signal_adj_pvalnumpy.ndarray, pd.DataFrame

      Benjamini-Hochberg adjusted p values for each cell in the step 5.

  • If method is “boxplot” returns MDDCBoxplotResult:
    • signalnumpy.ndarray, pd.DataFrame

      Matrix indicating significant signals with count greater than five and identified in the step 2 by the Monte Carlo method. 1 indicates a signal, 0 indicates non-signal.

    • corr_signal_pvalnumpy.ndarray, pd.DataFrame

      p-values for each cell in the contingency table in the step 5, when the \(r_{ij}\) (residual) values are mapped back to the standard normal distribution.

    • corr_signal_adj_pvalnumpy.ndarray, pd.DataFrame

      Benjamini-Hochberg adjusted p values for each cell in the step 5.

Notes#

This chunk_size option of the function is designed to be used in scenarios where the contingency table dimensions and the number of Monte Carlo replications are large. In such cases, the Monte Carlo samples need to be generated sequentially to ensure that the memory footprint remains manageable and the generated samples fit into the available RAM.

References#

Liu, A., Mukhopadhyay, R., and Markatou, M. (2024). MDDC: An R and Python package for adverse event identification in pharmacovigilance data. arXiv preprint. arXiv:2410.01168

Contents