install.packages("tidyverse")
install.packages("meta")
install.packages("metafor")
devtools::install_github("MathiasHarrer/dmetar")
# if not working, clone the package from github and unzip and install from local
devtools::install("C:/dmetar-master/dmetar-master")
B. Error
Error: (converted from warning) cannot remove prior installation of package ‘digest’
C. Workaround
get library location: Sys.getenv(“R_LIBS_USER”)
close R program completely
Go to R library: C:\Program Files\R\R-3.6.1\library
Delete “digest” folder manually
Rerun R with above code and the error message will not appear again and the dmetar package will be successfully intstalled.
4. r = .27 for binary variable and continuous variable
2*0.27/ sqrt(1-0.27^2) ES=0.560829 formula from D.W. Wilson’s slides
5. Group / Mean test scroe / n 1 / 55.38 / 13 2 / 59.40 / 18 3 / 75.14 / 37 4 / 88 / 22 F(3,86) = 7.05, for meta-analysis only interested group 1 and 2, std not reported.
Standardized Mean Difference (d) F-Test, 3 or more Groups d= -0.1658
6. 2 x 2 table n / group / % not improved / % improved 42 / T / 32% / 68% 29 / C / 37% / 63%
Standardized Mean Difference (d) Frequency Distribution (Proportions) d= 0.1057
7. frequency table for T and C group, 60 cases for each group (no report of the means and stds) Degrees of Condition / T(n) / C(n) 0 / 15 / 20 1 / 15 / 20 2 / 15 / 10 3 / 15 / 10
Standardized Mean Difference (d) Frequency Distribution d = 0.305
8. regression analysis with nonequivalent comparison group design covariates: employment status, marital status, age etc. treatment: intervention / probation only unstandardized regression coefficient: -.523 std for DV, severity of physical abuse: s=9.23 sample size (intervention / probation only) : n1= 125 / n2=254
Example 1: Means and Standard Deviation Example 2: T-Test, Unequal Sample Size Example 3: T-Test P-Value, Unequal Sample Size Example 5: F-Test, 3 or more GroupsExample 6, Frequency distribution (Proportions)Example 7, Frequency distributionExample 8, Unstandardized Regression Coefficient
D. Citation
Wilson, D. B. (date of version). Meta-analysis macros for SAS, SPSS, and Stata. Retrieved November 3, 2019, from http://mason.gmu.edu/~dwilsonb/ma.html
Group means, SD, N, estimated effect size, appropriate SD, F, t test, significance, inverse variance weight
D. Statistics
Cohen’s kappa
Cohen’s d
effect size
aggregate/weighted mean effect size
95% confidence interval: upper and lower
homogeneity of variance (Q statistic): Test if the mean effect size of the studies are significantly heterogeneous (p<.05), which means that there is more variability in the effect sizes than would be expected from sampling error and that the effect sized did not estimate common population mean (Lipsey & Wilson, 2001)
df: degrees of freedom
I square (%): the percentage of variability of the effect size that is attributable to true heterogeneity, that is, over and above the sampling error.
Outlier detection
mixed-effects model (consider studies as random effects): moderator analysis for heterogeneity (allow for population parameters to vary across studies, reducing the probability of committing a Type I error)
Proc GLM/ANOVA (consider studies as fixed effects): moderator analysis for heterogeneity
Region
Socioeconomic status
Geographical location
Education level
Setting
Language
sampling method
Statistical difference in the mean effect size of methodological feature of the study
confidence in effect size derivation (medium, high)
reliability (not reported, reported)
validity (not reported vs. reported
classic fail-safe N/Orwin’s fail-safe N: The number of missing null studies needed to bring the current mean effect size of the meta-analysis to .04. Threshhold is 5k+10, k is number of studies for the meta-analysis. If the N is greater than the 5k+10 limit then it is unlikely that publication bias poses a significant threat to the validity of findings of the meta-analysis.
Used to assess publication bias. eg. control for bias in studies (tightly controlled, loosely controlled, not controlled)
E. Purpose/Research Questions
Whether the treatment is associated with single effect or multiple effects?
Understand the variability of studies on the association of treatment with single or multiple effects, and explain the variable effects potentially through the study features (moderators). How do the effects of the treatment vary different study features?
F. Reference
Odesope et al, 2010: A Systematic Review and Meta-Analysis of the Cognitive Correlates of Bilingualism
SUGI27 paper: (Hamer and Simpson, 2002) SAS Tools for Meta-Analysis* *The methodology in this paper is ok, but the example was not interpreted correctly. I have corrected the example in this post.
A meta-analysis is a statistical analysis that combines the results of multiple scientific studies. Meta-analysis can be performed when there are multiple scientific studies addressing the same question, with each individual study reporting measurements that are expected to have some degree of error. The aim then is to use approaches from statistics to derive a pooled estimate closest to the unknown common truth based on how this error is perceived.
Research collect data for meta-analysis by systematic review of the literature in the field, and compile data directly from the summary statistics in the publication.
C. Problem with simply lumping the data from different studies together
Not consider treatment-by-study interaction
Assume response rates are the same in all studies.
D. SAS Solution (follow Hamer and Simpson’s paper, but corrected the output from the paper)
Create data set with the results of 2 studies. B: Remitted; N:Not remitted; P: Placebo; D: Drug.
I have used B (Better) to indicate Remitted cases because Proc Freq test is based on column 1 and row 1 of the 2 by 2 table, so if we code R for Remitted cases then the remitted case will be in column 2 because the table is by alphabetical order and R is after N.
The Hamer and Simpson paper actually tested the null hypothesis for the non-effective cases rather than the effective cases.
data chm;
input study $ response $ trt $ cellfreq @@;
datalines;
study1 B P 24 study1 N P 3
study1 B D 58 study1 N D 30
study2 B P 16 study2 N P 57
study2 B D 2 study2 N D 10
;
run;
Run Cochran-Mantel-Haenszel Statistics using Proc Freq procedure with cmh option.
The Mantel-Haenszel estimator of the common odds ratio assumed the estimation to be homogeneous among both studies.
The Mentel-Haenszel statistics tests the null hypothesis that the response rate is the same for the two treatments, after adjusting for possible differences in study response rates.
For Proc Freq testing options, make sure the group that you want to tested are in row 1 and column 1. It is also important to crosstab treatment as row and response as column, so the interpretation of the relative risk for the risk of improvement make sense. In Hamper and Simpon’s paper the crosstab has been transposed, therefore the relative risk output doesn’t make sense.
G. Interpretation
The CMH test statistics is 4.65 with a p-value of 0.03, therefore, we can reject the null hypothesis that there is no association between treatment and response. P-value lower than 0.05 indicates that the association between treatment and response remains strong after adjusting for study.
Relative Risk (Column 1) equals to 0.74 which means the probability of the improvement with the drug is 0.74 time the probability of the improvement with the placebo.
Relative Risk (Column 2) equals to 1.51 which means the probability of no improvement in the symptoms with the drug is 1.51 times the probability of no improvement with the placebo.
The Breslow-Day test has a large p-value of 0.295 which indicates there is no significant difference in the odds ratios among the studies.
* I will show the odds ratio and relative risk calculation in Excel in another post.
