PHX Template v1.4.3.1 for the EMA’s ABEL

Phoenix Certara Forums Phoenix WNL basics
1

Dear all,

I updated the template for the EMA’s reference-scaling method “Average Bioequivalence with Expanding Limits – ABEL”.

New or changed:

  • Supported designs:

    • Partial replicate design with sequences RRT|RTR|TRR.1
    • Full replicate design with sequences RTRT|TRTR.
    • Full replicate design with sequences RTTR|TRRT (new).
    • Full replicate design with sequences RTR|TRT (new).
    • Full replicate design with sequences RTT|TRR (new).

  • Not supported designs (and IMHO, should never be):

    • Extra-reference design with sequences RTR|TRR.
      Reason: Since T is not administered in period 3 the models will give a biased treatment estimate in the presence of period effects.
    • Balaam’s design with sequences RT|TR|RR|TT.
      Reason: Very poor power characteristics (sample size ~8× of a 4-period replicate and ~5× of a 3-period replicate). If limited sampling volume is an issue, consider one of the 3-period designs instead.
    • Full replicate design with sequences RTTR|TRRT|RRTT|TTRR. For drawbacks of this design see the FDA’s Guidance for Industry: Statistical Approaches to Establishing Bio ­equi ­va ­lence, Appen ­dix B (2001).

  • The intra-subject CV(s) are reported to two digits (previously rounded to one digit as in the Q&A document) and used in full precision.

  • The expanded limits above the scaling cap of CVwR 50% are no more hard coded to 69.84–143.19% but calculated in full precision according to
    100exp( ±0.76sqrt(ln(0.5^2+1))).
    Reason: Rounded limits are not symmetrical around 100% and lead to discontinuities…

    DiscHi.png
    DiscHi.png374×305 2.62 KB

    DiscLo.png
    DiscLo.png375×301 2.34 KB

    The inclusion of the 90% CI is still assessed based one the two-digits rounded percentages according to the guideline. The same holds for the GMR-restriction (within 80.00–125.00%).

  • Added a data wizard Design to the sub-workflow Prepare Dataset for Analysis:

    • Checks whether the design is a partial replicate.1
    • Checks whether the design is one of the 3-period full replicates (RTR|TRT or RTT|TRR).

#1 prevents the meaningless2 CVWT in ‘Method C’ to be shown in the Table Final Conclusion.

#2 will be used ‘downstream’ to assess whether at least 12 subjects are in sequence RTR in the RTR|TRT-design (required for a „reliable estimate of the CV*wR**”* according to the Q&A document Rev. 12 of June 2015). Note: In a pro ­perly powered study such an outcome is not realistic anyway (more than 42% dropouts).
Although not mentioned in the Q&A document, the same assessment is per ­formed for sequence TRR in the RTT|TRR-design.

  • Sub-workflow Analysis|Standard Average BE|Other Methods object BE Method B log changed the Degrees of Freedom to ⦿ Residual in com ­pli ­ance with the Q&A – expected to be equivalent to SAS DDFM=CONTAIN (new in v1.4.3.1).
  • Sub-workflow ABEL|Calculation Steps
    • For validation purposes the Data Wizard Assessment gives the degrees of freedom of the difference and the log halfwidth (new in v1.4.3.1).
    • If the Table Final Conclusion is executed, the calculation of all Methods (A, B, and C) is triggered.
    • The Table Final Conclusion has additional columns:
      90% CI (pass|fail), GMR (pass|fail), and overall (pass|fail).
    • If the design was RTR|TRT or RTT|TRR, an addi ­tio ­nal column Reliable estimate of CVwR (yes|no) will be given right to the column CVwR (%).
    • If the design was a partial replicate1 the column CVwT (%) will not be displayed since due to the over-spe ­cified model the value esti ­mated by ‘Method C’ is meaning ­less.2
  • Sub-workflow ABEL|Outlier Analysis
    • The filter for sequences and periods was hard-coded for RTRT|TRTR and RRT|RTR|TRR. There ­fore, RTR|TRT and RTTR|TRRT did not work. Now the work ­flow accepts any of the supported designs.
    • Nit-picky: Changed the outlier-flag from ±2 to ±1.959964.
    • Removed the 3px-border of the Box Plot.
    • PHX7-version of the template: Increased the Title Area Size of the Box Plot from 30 to 40.
  • Documents folder.
    • Added the current version of the EMA’s Q&A document.
    • Updated the Instructions.

The template was developed in PHX 6.4.0.768 and tested in PHX 7.0.0.2535 as well. The template was cross-validated with 11 data sets (all designs, up to 360 subjects) against SAS 9.4 (Methods A, B, C), R 3.3.2 (A, B), STATISTICA (A), and partly (Q&A data sets I and II; Methods A, B) against STaTa 14, SPSS 22, and JMP 10. As usual, use at your own risk.

If you become aware of any defects, please report here.

[hr]Comments:

  • The partial replicate is – in statistical terms – a lousy design. If you want to avoid trouble, please forget it or report here why you want to use it.
  • ‘Method C’ is an over-specified model. It is essentially forcing the soft ­ware to esti ­mate the within-sub ­ject vari ­ability of T, which the data cannot provide (since T was not repeated!). Hence, Phoenix – cor ­rectly – throws a warning:
    Newton’s algorithm converged with modified Hessian. Output is suspect.
    Model may be over-specified. A simpler model could be tried.
    Similar in SAS:
    Convergence criteria met but final hessian is not positive definite.
    F.i. the Q&A data set II gives a CVWT 8.65% (Phoenix) or 3.87% (SAS). Both are meaningless.

[hr]I removed the 1.4.3 templates from this post. Please download v1.4.3.1 provided in [post=‘4244’]this post[/post] or in the previous versions.use the workaround as described.

2

Thanks for the early Christmas presents Helmut ;0)

Unfortunately some other projects will prevent me unwrapping them to look at for a few weeks!

Simon.

3

Hi Simon,

Welcome!

When I tried to shrink the size of the file (export template/clearing the history and import again) I made a mistake. The workflow Analysis of the 3 Dec. versions was a sub-workflow of Prepare Dataset for Analysis

Good.png
Good.png260×146 3.7 KB

and not as described in the instructions (and as it was in previous versions):

Better.png
Better.png260×146 3.72 KB

[post=‘4220’]Corrected[/post] in the meantime.
If someone downloaded the versions of 3 Dec. already, please use the updated versions instead or:

  • Drag/drop Analysis before Prepare Dataset for Analysis.
  • Drag/drop Prepare Dataset before Analysis before Analysis.

In actual projects delete Example Data EMA 618604 2008 Rev 7 from the Data folder and the PDFs in the Documents folder in order to minimize the size of the file.

4

Hi Helmut!

excellent templates, thank you!

BR,

Mittyright

PS I removed my remark regarding design object, everything is fine there

5

Hi Simon,

Maybe this is a Trojan Horse…

The EMA’s Q&A document gives this SAS-code for Method B:

proc mixed data=replicate;
  class formulation subject period sequence;
  model logDATA= sequence period formulation;
  random subject(sequence);
  estimate "test-ref" formulation -1 1 / CL alpha=0.10;
run;

Since no method for the DFs is specified in the model statement, SAS falls back to its default, which is

DDFM=CONTAIN

Splendid. In Phoenix We have Satterthwaite (default) and Residual.

Currently we are working on cross-validation of different software packages. Our target is agreement of the CVwR and the 90% CI in percent, rounded to two decimals (according to the GL).
In one of our test data sets (RTRT|TRTR, unbalanced 38|39, incomplete period data 0|1|7|2, extreme CVwR 222%) we discovered the first disagreement in 90% CI:

SAS (contain): 90.34 – 157.88

R package nlme: 90.34 – 157.88

R package lmer (Satt.): 90.35 – 157.88

Phoenix (Sattertwaite): 90.35 – 157.88

Phoenix (Residual): 90.34 – 157.88

AFAIK, nlme was designed to reproduce SAS DDFM=BETWEENWITHIN. Duno.

See the attached comparison of both DF-options in Phoenix. As expected DFs are identical if the design is balanced and the data complete. However, I don’t see a consistent pattern for the other data sets.

  • In two cases Satterthwaite is more conservative than Residual (DFs lower).
  • In five cases it is the other way ’round.
  • Interesting the unbalanced DS11: identical DFs.

If we want to take the Q&A literally, Satterthwaite’s DFs should not be used.
Any ideas? Change the DF option in the template to Residual? Is PHX’s Residual equivalent to SAS’s DDFM=CONTAIN (and why not DDFM=RESIDUAL)?

[hr]Edit: Here is a comparison of Phoenix’ Degrees of Freedom with SAS (THX to Detlew Labes!):
Phoenix SAS
───────────── ───────────────────
Satt. Residual SATT CONTAIN RESIDUAL
─────── ──────── ─────── ─────── ────────
DS01 216.939 217 216.939 217 292
DS02 45 45 45 45 66
DS03 143.267 143 143.267 143 218
DS04 99 99 99 99 147
DS05 74 74 74 74 98
DS06 216.939 217 216.939 217 292
DS07 717 717 717 717 1074
DS08 662 662 662 662 882
DS09 662 662 662 662 882
DS10 33 33 33 33 49
DS11 107 107 107 107 142
DS12 291.173 217 291.173 217 292
DS13 554.657 550 554.657 550 770
DS14 197.440 192 197.440 192 267
DS15 545.657 550 545.657 550 770

Hence, I believe that Phoenix’ Residual Degrees of Freedom equal SAS DDFM=CONTAIN. I attached the updated templates. If you want to take the EMA’s Q&A literally in previous versions: Navigate in the sub-workflow Analysis|Standard Average BE|Other Methods to object BE Method B log and change in the tab General Options > Degrees of Freedom from ⦿ Satterthwaite to ⦿ Residual.
Note that Satterthwaite’s DFs can be lower/equal/higher than the Residual DFs. This has consequences on the CI. In our data sets we had balanced (equal number of subjects in each sequence) and unbalanced (unequal number of subjects / sequence) ones, complete (data of all periods available) and incomplete (some period data missing) ones, and all of their combinations. An example of an unbalanced and incomplete data set is DS01 given in the Q&A. In comparing the outcome based on Satt. and Residual I observed this pattern of the CI:
balanced + complete: identical
unbalanced + complete: identical
balanced + incomplete: liberal
unbalanced + incomplete: liberal or conservative

MethodB_and_PhoenixDFs.xls (33 KB)EMA ABEL Project Template v1.4.3.1 PHX7.0.phxproj (3.36 MB)EMA ABEL Project Template v1.4.3.1 PHX6.4.phxproj (3.25 MB)