|
 一、Critical appraisal of literatures之意義
Critical appraisal 乃是在有限的時間內評估文獻的良窳及其可利用性。
因此，下列原則應能掌握：
- 是否知道如何評讀？且其評讀方法相當一致？
何種工具或項目可供利用？
- 評讀應包括那兩大重點？
(internal) validity → 文獻品質
(external) validity ＝ application → 文獻結果能否應用於特定病人
- 是否能對每篇文獻充分且有效地利用類似方法加以評讀？
（觀察其平日之評讀文獻之功夫）
- 是否有其獨到的心得及方式？（加分部分）
其終極目標則是在15~30分鐘內能初步判斷所搜尋到的文獻是否合用，是否有詳讀之價值。
二、Critical Appraisal of a Randomized Controlled Trial (RCT)
Critical appraisal is the process of systematically examining research to judge its validity, results and relevance in a particular context before using it to inform a decision. It is an essential skill because it allows physicians to find and use research evidence reliably and efficiently. Critical appraisal of a RCT is one of the basic elements of epidemiology and evidence-based medicine. Various frameworks and guides for critical appraisal currently exist. Critical appraisal of RCTs include assessing the methodological quality of the study, examining how the design, conduct and analysis of the RCT minimize biases (e.g., selection, measurement) and generalizability of the study results. Important elements for critical appraisal of randomized controlled trials for therapy are summarized as follows:
- Are the results of the study valid? (Validity)
(A)Did experimental and control groups begin the study with a similar prognosis?
- Were participants randomized and randomization concealed?
- Were participants analyzed in the groups to which they were randomized? (Intention to treat)
- Were participants in the treatment and control groups similar with respect to known prognostic factors (not whether there are statistically significant differences in known prognostic factors between groups, but rather, the magnitude of these differences)?
- Were groups treated equally, apart from the experiment therapy?
- Did the study have enough participants to minimize the play of chance (sample size estimation)?
- Were the outcomes clearly defined?
- What were the adherence rate and dropout rate?
(B)Did experimental and control groups retain a similar prognosis after the study started?
(1)Were participants, clinicians and outcome assessors aware of group allocation?
(2)Was follow-up sufficiently long and complete?
(C)Is there any factor influencing the validity of the study? (e.g., biases)
(D)Is there any issue in statistical analyses?
- What are the results? (Reliability)
(1) What is the magnitude of the treatment effect? ( i.e., number needed to treat, relative risk, absolute risk reduction, relative risk reduction, etc)
(2)What is the precision of the treatment effect ( i.e., 95% confidence interval)?
(3)Do you believe the results and what are your reasons?
- How do I apply the results to patient care? (Applicability)
(1)Were the study patients similar to my patient?
(2)Were all clinically important outcomes considered?
(3)Are the likely treatment benefits worth the potential harm and costs?
(4)Conclusion (Feedback to the patient)
References:
- Guyatt GH, Sackett DL, Cook DJ. Users' guides to the medical literature. II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients? Evidence-based medicine working group. JAMA 1994; 271(1): 59–63.
- Greenhalgh T. How to read a paper. Getting your bearings (deciding what the paper is about). BMJ.1997;315(7102):243-246.
- Gordon Guyatt, et al. Users' guides to the medical literature : a manual for evidence-based clinical practice. 2nd ed.McGraw-Hill Medical, 2008.
- Elwood JM. Critical Appraisal of Epidemiological Studies and Clinical Trials. 3rd ed. Oxford: Oxford University Press, 2007.
- Critical Appraisal Skills Programme (CASP): http://www.phru.nhs.uk/Pages/PHD/resources.htm
- Scottish Intercollegiate Guidelines Network (SIGN): http://www.sign.ac.uk
三、Scoring worksheet in EBM campaign(2009)
Items |
Score |
1. |
Are the results valid? |
|
|
(A) Did experimental and control groups of the study begin with a similar prognosis? |
|
|
(1)Were participants randomized and randomization concealed? ( 1a ) |
5 |
|
(2)Were participants analyzed in the groups to which they were randomized? (Intention to treat) ( 1b ) |
5 |
|
(3)Were participants in the treatment and control groups similar with respect to known prognostic factors (not whether there are statistically significant differences in known prognostic factors between groups, but rather, the magnitude of these differences)? ( 1c ) |
10 |
|
(4)Were groups treated equally, apart from the experiment therapy? ( 1d ) |
5 |
|
(5)Did the study have enough participants to minimize the play of chance (sample size estimation)? (1e ) |
5 |
|
(6)Were the outcomes clearly defined? ( 1f ) |
5 |
|
(7)What were the adherence rate and dropout rate? ( 1g ) |
5 |
|
(B) Did experimental and control groups retain a similar prognosis after the study started? |
|
|
(1)Were participants, clinicians and outcome assessors aware of group allocation? ( 2a ) |
5 |
|
(2)Was follow-up sufficiently long and complete? ( 2b ) |
5 |
2. |
What are the results? |
|
|
(1) What is the magnitude of the treatment effect? ( i.e., number needed to treat, relative risk, absolute risk reduction, relative risk reduction) ( 3a ) |
5 |
|
(2) Is there any statistical issue? ( 3b ) |
5 |
|
(3) What is the precision of the treatment effect ( i.e., 95% confidence interval)? ( 3c ) |
5 |
3. |
Other aspects that are not mentioned above ( 4a ) |
5 |
4. |
General conclusion (Do you believe the results? Why?) ( 5a ) |
10 |
5. |
How can I apply the results to patient care? |
|
|
(1) Were the study patients similar to my patient? ( 6a ) |
5 |
|
(2) Were all clinically important outcomes considered? ( 6b ) |
5 |
|
(3) Are the likely treatment benefits worth the potential harm and costs? ( 6c ) |
5 |
|
(4) Conclusion (6d ) |
5 |
Total Score |
100 |
四、實證醫學Critical Appraisal的評分標準
原則1：文字上有提到這個項目，才給分，至於順序先不考量。
原則2：重視結構及名詞的用法。雖然意思好像有點到，但是格式或表達方式不對，仍不給分。
Randomization and concealed |
1a |
5 |
有完整說明這研究是否是RCT有沒有conceal才給分。 |
Intention to treat |
1b |
5 |
有提到，且正確給全部的分數，否則給三分，沒提到，得零分。 |
Same background between groups |
1c |
10 |
有提到，且正確給全部的分數，否則按照完整性給七分、五分、三分，沒提到，得零分。 |
Equal treatment |
1d |
5 |
有提到，且正確給全部的分數，否則給三分，沒提到，得零分。 |
Sample size |
1e |
5 |
有提到，且正確給全部的分數，否則給三分，沒提到，得零分。 |
Outcome clear |
1f |
5 |
有提到outcome的標準，且正確給全部的分數，否則給三分，沒提到，得零分。 |
Adherence and dropout |
1g |
5 |
有提到dropout、是否影響結果，且正確，給全部的分數，否則給三分，沒提到，得零分。 |
Blindness |
2a |
5 |
有提到，且正確給全部的分數，否則給三分，沒提到，得零分。 |
Enough follow up |
2b |
5 |
有提到，且正確給全部的分數，否則給三分，沒提到，得零分。 |
Magnitude of Treatment effects |
3a |
5 |
有提到，且正確（對於治療結果的統計結論）給全部的分數，否則給三分，沒提到，得零分。 |
Statistical Issue |
3b |
5 |
有提到，且正確給全部的分數，否則按照完整性給三分，沒提到，得零分。 |
Precision |
3c |
5 |
有提到（大致上是信賴區間的解讀），且正確給全部的分數，否則給三分，沒提到，得零分。 |
Other aspects |
4a |
5 |
有提到且正確給全部的分數（例如conflict of interest等），否則按照完整性多寡給三分，沒提到，得零分。 |
General consideration |
5a |
10 |
綜合評論這研究的品質、與PICO的關連、是否能解答我們的問題等，有提到，且正確給全部的分數，否則給七分、五分、三分，沒提到，得零分。 |
Similar patients |
6a |
5 |
有提到（這研究與我們的病人是否類似），且正確給全部的分數，否則按照完整性給三分，沒提到，得零分。 |
All outcome considered |
6b |
5 |
有提到（這研究與我們的病人所考量的『結果』是否類似），且正確給全部的分數，否則按照完整性給三分，沒提到，得零分。 |
Benefit consideration |
6c |
5 |
有提到（這治療所帶來的效益與花費及危險性等），且正確給全部的分數，否則按照完整性給三分，沒提到，得零分。 |
Conclusion |
6d |
5 |
有提到綜合的結果完整回饋到這臨床情況，不只正確而且是用病患能理解的話來說，給全部的分數，否則按照完整性給三分，沒提到，得零分。 |
五、審查心得報告
前言：
此次文獻批判(Critical Appraisal)選讀文章是由GISSI-HF investigators 在Lancet 2008; 372: 1223–30發表的文章〝Effect of n-3 polyunsaturated fatty acids in patients with chronic heart failure (the GISSI-HF trial): a randomised, double-blind, placebo-controlled trial〞，經由內科學會EBM小組成員改寫成模擬文章，並在內容作一稍許的更動，更動的地方包括下列幾處：
- 把藥物改成peanut butter。
- 把研究地區改成台灣，並改名為GIGI-FF。
- 把Blinded的機制刪除。
- 把如何randomly assigned的方法消除，原文是以concealed，computerized telephone randomization system，並且是以stratified by site的randomization方式。
- compliance的測量是以服用at least 80% of the day of observation。
- 對outcome events的評判是blindly，並且以an ad-hoc committee on the basis of pre-agreed definitions and procedures進行。
- 對primary及secondary endpoint outcomes的定義刪除。
- 把sample size及power的計算部分刪除。
- 把統計檢定對proportional hazard假設的部分刪除，其他則保持原來的風貌。
審查心得：大多數參賽者均能針對內容設計作一清楚的描述，也對結果有自己的看法，但仍有一些問題有待討論：
- 有些參賽者使用制式的樣式及格式，如Oxford、JAMA等的條例式範本，但未能把自己的意見納入，只單獨註明Yes，No，Unclear。
- 參賽者對文章刪除的部分均能指出缺乏randomization的說明，但對compliance及outcomes的認定及sample size，power的缺乏幾乎完全付之闕如，可能是在條例式範本中並未提及此細項的檢查。
- 大多參賽者花費篇幅在計算其治療效果的統計量，如ARR、IRR、NNT，甚至有提供其95% CI值的計算公式，明顯可知道參賽者事先已準備好運算的公式。但事實上，論文中已提供NNT等統計量，參賽者只要引用即可，不需花費篇幅及時間再作double-check的工作。
- 本研究另外以statin的治療與否的設計也令很多參賽者混淆，此乃是因素設計(factorial design)的特性，此種factorial design是取其RCT的方便性，在同一樣本之下同時作兩種藥物的治療測試，前提是此兩種藥物不能有交互作用，很多參賽者均以為statin的使用對peanut的效果會有影響，惟作者已交待並無交互作用，故應可放心。
- 很多參賽者提出一些疑問，包括：
- peanut的口味如何作到double-blind
- 成本效益分析資料的缺乏
- Conflict of interest/informed consent
- 藥物安全性及醫療分佈等，值得鼓勵。
總結而言，大多數參賽者均能提出他們/她們自己的看法，利用一些EBM常用的工具來幫忙回答相關的問題，值得鼓勵及讚揚。 |