Key Features Testing for Clinical Reasoning

2013/12/12
2
Objectives for this presentation
講演で学んでいただきたいこと
KEY FEATURES TESTING FOR
CLINICAL DECISION MAKING
臨床推論のテスト：KEY FEATURES
• Be able to explain what a key features question is.
Joyce Pickering, MD, FRCPC, FACP
IRCME Visiting Professor 2013/2014
Dec. 12, 2013
• Demonstrate an understanding of how their validity
Key Features (KF)が何であるかを説明できる
• Be able to state why they were developed.
なぜ開発されたかを説明できる
has been tested.
妥当性がどう検証されてきたかを示す
• Be able to describe how to create a key features
question.
KF問題の作り方を説明できる
3
History 歴史
4
• G. Norman – studied patient management
problems. Concluded that NormanはPMPを評価
し以下の結論
• Trying to test テストしたいこと
• clinical reasoning skills 臨床推論スキル
• Problem solving skills 問題解決スキル
• Clinical decision making 臨床決断
• Problem solving skills are specific to the case or problem
encountered. 問題解決スキルは症例特異的
• Patient Management Problems PMP
• Used in the 1980s in the Canadian national licencing
exams 1980年代にカナダの医師国家試験で利用された
• Used a special pen to make a result visible
• E.g. if a CBC was highlighted with the pen, the results of the
CBC showed up なぞると文字が浮かび上がる特殊なペンを使う
• Poor correlation across cases (12 to 15 cases used in
the Canadian licencing exam). 症例間での成績の相関
は低かった
• These skills are contingent on the effective manipulation of a
few elements of the problem that are critical to its successful
resolution – “key features”. 問題の2、3の要素を効果的に扱
えるかどうかが、問題解決にとって決定的な意味を持つ（情報を
網羅しても、2、3の要素が扱えなければ問題解決できない）
• Therefore many more questions need to be asked (e.g 30 to
40) across many clinical areas in order to have a more
reliable assessment of clinical problem solving.
臨床問題解決に関し、より信頼性のある評価をするには、多くの
臨床領域にわたる問題が必要（例えば30~40問）
5
6
That is:
For example:
• You must ask about many different areas
多領域の問題が不可欠
• Doing well in clinical judgment in an obstetrics/gynecology
case does NOT mean you will do well in clinical judgement in
a pediatrics case.
• A KEY FEATURE may not be having a complete
産婦人科症例での正答は、小児科症例での正答の予測に役立たない
• However, the time to do the exam cannot be too long,
or it becomes very expensive.
試験時間が長すぎると高コスト
• Therefore we need to ask many short questions about many
areas.
よって多領域で多くの短問が必要
• These short questions need to focus on the most important
elements of a case – the KEY FEATURES.
differential diagnosis, but being able to think of the one
most dangerous possibility.
KFだけでは完全な鑑別診断に至らないこともあるが、最も危
険な可能性を思いつくことはできる
• A KEY FEATURE may be even if the candidate does not
know the diagnosis, they know that the situation is
probably dangerous and the patient must stay in the
hospital
KFは受験者が診断を知らなくても、状況が危険かもしれない
ので、院内に留まらせるという判断ができればよい
短問は症例の最も重要な要素に焦点を当てる→KEY FEATURES
1
2013/12/12
7
• These specific critical decision points are the KEY
8
• www.mcc.ca
FEATURES of the problem
これらのクリティカルな決断ポイントが問題のKFである
10
例題A
あなたは地域病院に赴任して間もない医師である。
小売業の55歳男性が、胸骨後面の痛みを訴えて来院。痛みは1年
前から反復し、労作で悪化し、中央、深部に
生じる。安静時には痛みはなし。頻度や強度は悪化なし。他に複
数の医師を受診したが、今回再度受診を希望。
喫煙歴は10本/日を20年間。
You are a physician who has recently come to practice in a
community hospital. A 55-year-old merchant, who has run his
own business for a number of years, comes to you with
retrosternal chest pain. This pain has been recurring in the past
year, is deep and central, and usually precipitated by exertion.
He has no pain at rest. Episodes have not become more frequent
or more severe. He has seen other physicians but he wishes a
reassessment at this time. The patient has a 20-year history of
smoking 10 cigarettes a day
質問１）
考えられる診断として最も可能性の高いものはどれか。
次の17の選択肢から一つだけ選びなさい。
1. 慢性膵炎
Chronic pericarditis
2. うっ血性心不全 CHF
3. 胸部大動脈解離
Dissecting thoracic AA
4. 過換気症候
Hyperventilation syndrome
5. 縦隔炎 Mediastinitis
6. 心筋梗塞 MI
7. 慢性心筋炎 Myocarditis
8. 筋筋膜疼痛症候群
Myofascial syndrome
9. 神経衰弱症 Neurasthenia
10. 冠攣縮性狭心症
Prinzmetal’s angina
11. 逆流性食道炎
Reflux esophagitis
12. 安定狭心症
Stable angina
13. T1-T6神経痛 Neuralgia
14. T1-T6神経根炎 Radiculitis
15. 慢性甲状腺炎 chr. thyroiditis
16. Tietze症候群
（反復性肋軟骨炎）
17. 不安定狭心症 unstable angina
11
質問２）
身体診察では、緊張感があるが、呼吸苦はなし。
チアノーゼ、顔面蒼白、黄疸も見られない。
BT 37℃
BP 140/95 mmHg
P 78/分, 整
RR 15/分
身長 170 cm
体重 98 kg
頸静脈胸骨角上3 cm
心音正常
呼吸音正常
腹部所見なし
末梢脈拍触知
浮腫なし
眼底正常
12
この患者に対する診断では、どの検査や処置を選択するか。
次の21の選択肢から適切なものをすべて選びなさい。
1. 冠動脈造影
coronary angiogram
2. room airの血液ガス ABG
3. バリウム嚥下による造影
barium swallow
4. 気管支鏡 bronchoscopy
5. 胸部X線 CXR
6. 心エコーechocardiogram
7. 血沈 ESR
8. 食道運動機能検査 eso.motility
9. 負荷ラジオアイソトープ血管造影
exercise radionucleotide angio
10. 運動負荷テスト EST
11. 空腹・随時血糖
fasting and spot glucose
12. 上部消化管内視鏡
gastroscopy
13. Hb、白血球数と分画
HGB.WBC
14. HIVスクリーニング
15. 血小板数 platelets
16. 安静時心電図 EKG
17. 血清コレステロール
（総、HDL、TG）
18. 血清CKとCK-MB
19. 血清蛋白電気泳 SPEP
20. タリウム心筋シンチ
stress thallium
21. 血清総蛋白とアルブミン
protein and albumin
2
2013/12/12
質問３
13
この患者に対し、問題が生じるリスクを下げるためにどのような
アドバイスをおこないますか。下記17の選択肢から6つまで選
びなさい。
1. 降圧療法 BP reduction
2. 血糖低減 bl.sugar reduction 10. 理学療法 physiotherapy
11. 心理療法 psychotherapy
3. 活動レベル低減
12. 呼吸機能検査 PFTs
decrease activity
13. 休養プログラム
4. 運動プログラム
relax. program
exercise program
14. 禁酒 stop alcohol
5. 食物繊維の多い食事
15. 禁煙 stop smoking
high fiber diet
16. ストレス低減
6. 高蛋白の食事
stress reduction
high protein diet
17. 減量 weight reduction
7. 催眠療法 hypnotherapy
8. 低コレステロール食 low chol. diet
9. 低蛋白、高炭水化物食
low prot. High carb diet
14
• 例題B
不安そうな若い母親が1歳3か月の娘を連れて救急外来受診。この幼
児は母親がジメンヒドリナート（制吐剤）とリンゴジュースを与えてから、
2日間下痢をしています。嘔吐はなし。具合が悪そうで、母親の腕の
中で泣いているが、流涙はなし。体力が落ち、気力の低下が見られる。
An anxious young mother brings her 15-month-old daughter to
the Emergency Department. The child has had diarrhea for 2 days
for which her mother gave her dimenhydrinate and apple juice.
The child has not vomited. She appears ill and is crying with no
tears in her mother's arms. She seems weak and lethargic.
バイタルサイン：
体温（腋下） 36.5℃
脈拍
120/分
呼吸
40回/分
血圧
85/55 mmHg
15
16
質問1）
この幼児のマネジメントはどうするか？（4つまで選択可）
体重11kg。流涙なし。粘膜乾燥あり。Capillary refill time
正常。皮膚ツルゴール低下し、腹部皮膚がテント状。おむ
つは少量の緑色の水様便がある以外は乾いている。
She weighs 11 kg. She has no tearing. Mucous
membranes are dry to the touch. The capillary refill
time is normal. There is diminished skin turgor with
some tenting of the abdominal skin. The diaper is
dry except for a small amount of watery green stool.
腹部レントゲン Abd. Xray
リンゴジュースを与え、飲み方を確認
動脈血ガス ABG
血糖 blood glucose
BUN
血算 CBC
気の抜けたジンジャーエールを与え、
飲み方を確認
flat ginger ale and observation
8. ブドウ糖水を与え、飲み方を確認
glucose water and observation
9. 5%のD-グルコースの静注 IV D5W
10. 5%のD-グルコース入り生食を静注
IV D5Nsal.
1.
2.
3.
4.
5.
6.
7.
経鼻胃水分補給 ORS by N/G
経口絶食 NPO
経口補水 ORS
血清クレアチニン creat
血清電解質 lytes
血清浸透圧 osmolal.
便培養 stool culture
便虫卵・寄生虫検査
stool O and P
19. 検尿 U/A
20. 尿中電解質 urine lytes
21. 尿浸透圧 U osm.
11.
12.
13.
14.
15.
16.
17.
18.
17
質問２）
今回、この赤ちゃんにはどのような薬を投与すべきか。
（薬剤の名前を２つまで挙げる。投薬不要の場合は“なし”と記入。）
１．
２．
• What medications should you prescribe for this girl at this time?
例題C
55歳男性が巻き爪から感染症を発症し来院した。
あなたは担当し、適切に処置した後、家族歴から、
この患者の父親が大腸がんのため60歳で死亡した
ことに気づいた。
A 55-year-old man presents with an infected
ingrown toenail. You treat this condition
appropriately, and then notice on the chart that his
father died of bowel cancer at 60 years of age.
3
2013/12/12
19
質問１）
この患者からさらに情報を引き出すのに最も重要な内容は
以下のうちどれか。（注意：16の選択肢から4つ選ぶこと）
母親の膀胱がん家族歴
mother with bladder cancer
10. 腺腫様ポリープの病歴 personal
history of adenomatous polyps
3.
11. 過敏性腸症候群の病歴
frequent formed BMs
4.
12. ジアルジア症の病歴
Hx of giardiasis
5.
13. 過形成性ポリープの病歴
Hx of hyperplastic polys
6.
14. 炎症性腸疾患の病歴 Hx of IBD
15. 座り姿勢の多いライフスタイル
7.
sedentary lifestyle
8.
16. 喫煙習慣 smoking
What additional information would be most important to elicit from this patient?
アルコール依存症 alcoholism
兄弟の大腸がん家族歴
brother with bowel cancer
魚のくん製をよく食べるかどうか
diet high in smoked fish
父親ががんと診断された際の年齢
fathers age at diagnosis of cancer
高脂肪で繊維の少ない食生活
high fat low fiber diet
高タンパクで繊維の多い食生活
high protein high fiber diet
乳糖不耐症 lactose intolerance
下剤濫用 laxative abuse
1.
2.
9.
20
質問２）
この患者に推奨されることは次のうちどれか。
（注意：12の選択肢から１つ選ぶこと）
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
S状結腸鏡検査 60 cm sigmoidoscopy
CEA検査
全結腸内視鏡検査 colonoscopy
腹部CT abdominal CT
直腸診 digital rectal exam
注腸造影 double contrast barium enema
便潜血反応 fecal occult blood
遺伝子スクリーニング genetic screening
経静脈尿路造影 Intravenous pyelogram
腹部MRI abdominal MRI
腹部触診 abdominal palpation
腹部超音波検査 abdominal ultrasound
Which one of the following is your recommendation for this patient?
21
Objectives for this presentation
講演で学んでいただきたいこと
Have been used:
• Medical Council of Canada Qualifying Exam
• (national licencing exam)
• Be able to explain what a key features question is.
• College of Physicians and Surgeons of Pakistan
• America College of Physicians (MKSAP recertification)
• American College of Colon and Rectal Surgeons –
•
•
•
•
22
Key Features (KF)が何であるかを説明できる
• Be able to state why they were developed.
なぜ開発されたかを説明できる
CARSEP recertification
Royal Australian College General Practitioners
Swiss National Examination Board
McMaster clerkships
German Medical Schools
• Demonstrate an understanding of how their validity
has been tested.
妥当性がどう検証されてきたかを示す
• Be able to describe how to create a key features question.
KF問題の作り方を説明できる
23
• Key feature questions were developed and tested on final
year Canadian medical students ( 1987-91).
KF問題は、カナダの医学生最終学年に対して開発され、テス
トに利用されてきた。
24
• Face validity: Examinees were asked to judge the
importance of the clinical problems and the importance
and difficulty level of the competencies testing by the
questions.
表面妥当性：臨床問題の重要性、コンピテンシーのテストの
重要性や難易度について受験者に判断してもらった
• High levels of face validity
表面妥当性のレベルは高い
• 96% of students said the questions were at the “correct level”
although the average score on the exam was only 50%.
96％の学生は問題が適切なレベルだと答えたが、平均点は50%
4
2013/12/12
25
• Content validity: 内容妥当性
• MCC test committee members and 69 Cdn medical
faculty who were active in clerkship teaching asked to
judge retrospectively if they agreed with the key
features – 国家試験委員会とクラークシップ教育に関与す
る69名の教員がKFの一致度を判定
26
Canadian exams:
• Medical Council of Canada Qualifying Examination 1
(MCCQE1).
• Typically taken at the end of year 4 of medical school (graduation)
• 75% multiple choice questions (196 questions)
• 171 key features – 121 unanimous agreement, 158 > 80%
agreement.
171KFに対し、121は無記名で一致、158は8割以上の一致度
• 25% key features questions (Clinical Decision Making = CDM)
• Same judges asked to generate their own key features
for scenarios different from the first set, and these were
compared to the test key features. 同じメンバーに、異な
るシナリオに対するKFを各自作成してもらい、テストとの一
致度を判定
(80 questions)
4年生が受験。75%はMCQ（196問）。25%がKF問題（CDM－80問）
• Medical Council of Canada Qualifying Examination 2
(MCCQE2)
• Typically taken after 18 or 24 months of residency
• OSCE type examination (16 stations over two days)
研修開始後18、24カ月後受験。16ステーション、2日間のOSCE
• 94% level of concordance 94％の一致
• High level of content validity 内容妥当性のレベルは高い
27
28
Predictive validity 予測妥当性
• High correlation (0.8) between MCQ and CDM (key
• Study by Tamblyn et. al. JAMA, 2007
features) questions on the MCCQE1.
MCQとCDMは相関が高い（0.8程度）
• Physician scores on Canadian national exams as a
predictor of complaints to medical regulatory authorities.
Tamblynら（2007）が、医事規制当局へのクレームを目的変
数としてカナダ医師国家試験スコアの関与をみた
29
30
• Study population: 対象
• Independent variable: 説明変数
• Scores of candidates on these exams. 国家試験点数
• Dependent variable: 目的変数
• Retained complaints about these physicians, after they
began practice, to the regulatory body for the Province
of Ontario (College of Physicians and Surgeons of
Ontario) or to the regulatory body for the Province of
Quebec (Collège des médecins du Québec)
オンタリオ州、ケベック州の医療規制当局へのクレーム
• Population of Ontario about 9 million
• Population of Quebec about 7 million
• All physicians who took the MCCQE2 between 1993 and
1996 and were licenced to practice in at least one of Ontario
or Quebec = 3424 physicians. MCCQE2に合格した3424人の
医師
• All complaints filed with the provincial regulatory agency
from the date of licensure until 2005 were studied.
2005年までに免許取得した医師のクレーム
• Total 1116 complaints (4.9 complaints/100 practice years),
696 retained complaints 1116件（4.9件/100延べ診療年）、696
件は医師に問題ありと判定。
• 17.1% of physicians had at least one retained complaint.
17.1%の医師は少なくとも1回は問題ありのクレームを受けた
5
2013/12/12
31
32
Canadian exams:
問題のある
クレーム無しという測定値
Measured by lack of
complaints
• Medical Council of Canada Qualifying Examination 1
(MCCQE1).
• Typically taken at the end of year 4 of medical school (graduation)
• 75% multiple choice questions (196 questions)
• 25% key features questions (Clinical Decision Making = CDM)
??
Exam scores
(80 questions)
4年生が受験。75%はMCQ（196問）。25%がKF問題（CDM－80問）
• Medical Council of Canada Qualifying Examination 2
試験スコア
Quality of a
physician’s
practice
医師の診療の質
(MCCQE2)
• Typically taken after 18 or 24 months of residency
• OSCE type examination (16 stations over two days)
研修開始後18、24カ月後受験。16ステーション、2日間のOSCE
33
34
Associations with the number of retained complaints
score
RR 95% CI
P value
MCCQE 1 overall
1.39 (1.14-1.70)
.001
MCCQE1 MCQ
1.25 (1.03-1.50)
.02
MCCQE1 CDM
1.51 (1.25-1.84)
<.001
MCCQE2 overall
1.19 (1.00-1.42)
.05
MCCQE2 Communication 1.38 (1.18-1.62)
score
<.001
MCCQE2 data acquisition 0.98 (0.83-1.16)
score
.85
MCCQE2 Problem
solving score
1.02 (0.88-1.19)
0.78
MCCQE 2
Communication score
adjusted for CDM score
1.32 (1.13-1.71)
<.001
• Note: McMaster University study showed correlation of
multiple mini interview scores (for admission) with Clinical
Decision Making (CDM) scores.
McMaster大の研究では、入学試験でのMMIスコア とCDM
スコアも相関している
Change in relative rate per 2 SD decrease in score
35
36
Further evidence
• Article by Bordage, Meguerditchian and Tamblyn,
Academic Medicine 2013
• Defined physician-related practice indicators (PRINDS)
that, if not done, contribute to either causing or preventing
suboptimal care, and to what extent the two components
of the Medical Council of Canada exams measured these.
医師の診療の質指標（PRINDS）と医師国家試験スコア
• PRINDS defined via
• literature and Google search for practice errors,
• defined needed behaviour/skill, and then
• validated this with MCC tests committees and physician experts.
診療過誤、必要な行動やスキル、国家試験委員会や医師専門家による
妥当性検証によって、PRINDSは定義された
• Examples of PRINDS:
• Prescribe influenza vaccination for high risk patients.
ハイリスク患者へのインフルエンザワクチン接種
• Provide heart failure patients with written instructions on
activity, diet and medication
心不全患者への活動、食事、薬物療法に関する指導内容
記載
• Recognize an abdominal aortic aneurysm
• Recognize an ectopic pregnancy
腹部動脈瘤、子宮外妊娠の同定
6
2013/12/12
37
38
Objectives for this presentation
講演で学んでいただきたいこと
• Three parts of the exams looked at:
• Be able to explain what a key features question is.
• Part 1 Multiple choice questions
Key Features (KF)が何であるかを説明できる
• Part 1 Clinical Decision making (key features
• Be able to state why they were developed.
• Part 2 OSCE (entire exam)
• Part 1 Clinical decision making (key features) had
the highest proportion of marks related to
PRINDs (30%)
なぜ開発されたかを説明できる
• Demonstrate an understanding of how their validity has
been tested.
妥当性がどう検証されてきたかを示す
• Be able to describe how to create a key features
question.
KF問題の作り方を説明できる
CDMにはPRINDSと関連する問題が3割含まれていた
• Part 1 MCQ – 20%
• Part 2 Exam – 5%
39
40
Process:
• Define the domain 領域を決定
• What are the all skills that the candidate is expected to have?
受験者に持ってもらいたいスキルは何か
• The problems must be drawn from defined domain of problems for
which the candidate is accountable
受験者が対応可能な問題が及ぶ領域に基づいて問題を作成する
• Define the blueprint ブループリントを設定
• How much of each discipline should be on the exam?
各科の内容がどの程度試験に含まれているべきか
• The process of problem selection must be guided by an exam
blueprint
問題選択プロセスは試験ブループリントによって方向づける必要あり
• Write the scenarios and questions. シナリオと問
題の記載
• The exam should have about 40 of these scenarios.
試験には40問のシナリオが必要
• The questions should only be about the key features.
問題はKFに関する部分のみ
• Avoid “cueing” whenever possible.
可能な限り“手がかり”を避ける
41
Developing cases and questions
症例と問題の作成
42
Developing cases and questions
• Problem Selection and Context Definition
• Problem Selection and Context Definition
•
•
•
•
•
問題選定と状況の設定
Key Features definition KFの定義
Case scenario シナリオ
Test questions and format
テスト問題とフォーマットの設定
Scoring keys keyの点数
Pilot test the question 問題の試用
問題選定と状況の設定
• Key Features definition
• Case scenario
• Test questions and format
• Scoring keys
• Pilot test the question.
7
2013/12/12
43
Problem Selection and Context Definition
問題選定と状況の設定
• Select clinical problem: 臨床問題の選定
• Must be drawn from a list of problems that examinees
are responsible to know how to address
• Third trimester vaginal bleeding
• 受験者たちが対応法について知っているべき問題リストから選定
• 今回は妊娠後期不正出血
• Select age group: 年齢層の選定
• Pregnancy neonatal and infant; pediatrics; adolescent;
adult; geriatrics
• Pregnancy, neonatal and infant
• 妊娠・新生児・幼児、小児、思春期、成人、老年
44
• Select clinical situation and site of care
診療の設定
• Undifferentiated complaint; a single typical
problem; a multiple or multi-system problem; a
life-threatening event; preventive care and
health promotion.
• A life threatening event
• 分類不能の主訴、単一の典型的問題、多数・多系
統の問題、生命に危険が及ぶ状態、予防医学や健
康増進
• 今回は生命に危険が及ぶ状態
• 今回は妊娠・新生児・幼児
45
46
Medical Council of Canada Objectives
Developing cases and questions
• http://apps.mcc.ca/Objectives_Online/
• Problem Selection and Context Definition
• Key Features definition
KFの定義
• Case scenario
• Test questions and format
• Scoring keys
• Pilot test the question.
47
Key Features Definition KFの定義
• Ask this question: この質問をしてみる
• What are the critical steps in the resolution of this problem for
patients in this age group?
• この年齢層の患者における問題の解決にクリティカルな段階とは
• Where’s the challenge for a graduating medical student?
• 医学部卒業時点の学生がちょうど悩むのはどのあたりか
48
Key Features Definition
• The development of good key features can be
challenging. よいKFの開発は難しい
• Needs to be done by clinicians 臨床家が設定する必要性
• Discussion/review with colleagues is highly recommended
同僚とのブラッシュアップが強く推奨される
• Each key feature has three basic elements:
• If the critical issue is interpreting the data, then give the clinical data
• 各KFに３つの基本要素
• A set of conditions: “Given a pregnant woman experiencing third
trimester bleeding with no abdominal pain”
• A task: “consider placenta previa as a leading diagnosis”
• Some qualifier: “leading diagnosis”
• 状況「妊娠後期の腹痛のない出血」、課題「前置胎盤を主診断と考慮」、
KFのポイント「主診断」
and ask questions aimed at interpretation.
臨床データの解釈が課題なら、データを示して解釈を問う質問をする
• If the critical issue is one of eliciting the data, then give a patient
complaint and ask the examinee to identify the specific data that
would be elicited from the patient.
情報収集が課題なら、患者の主訴を提示し、患者から収集すべき特定
のデータを同定するような質問を受験者にする
8
2013/12/12
49
50
Key Features Definition
Key Features Definition
• In diagnosis 診断
• If the key feature is focusing on eliciting a probable
diagnosis, then ask for a differential diagnosis.
KFが可能性のある診断を挙げることであれば、鑑別診断を
尋ねる
• If the key feature is focusing on eliciting THE working
diagnosis or leading diagnosis at this time then ask for
one diagnosis.
KFが主たる診断を引き出すことであれば、一つだけ診断を
尋ねる
• Key feature steps or actions need to be a specific
as possible – KFのステップやアクションは可能な限
り特定的に
• E.g. prescribe morphine 例：モルヒネ処方
• E.g. order a complete blood count 例：全血球血算の指示
• Avoid: 避けるべき表現
• “manage appropriately” 適切にマネージする
• “investigate appropriately” 適切に検査する
51
52
Key Features Definition
Key Features Definition
• E.g. Given a pregnant woman experiencing third trimester
• Need to be developed locally, with clinicians familiar with
vaginal bleeding with no abdominal pain, the candidate
will: 妊娠後期に腹痛のない不正出血を経験している妊婦を
診たとき、受験者は・・・
• 1. Consider placenta previa as a leading diagnosis
前置胎盤を主たる診断とする
the medical student “clinical reality”. 各国や地域に応じた
形で医学生の臨床の現状を踏まえて開発される必要あり
• E.g. do you expect all your graduating medical students to be able
to recognize the clinical presentation of a placenta previa?
前置胎盤の臨床を全ての医学生が認識していると期待できるか
• Possible reference: Key Features for many clinical
• 2. Avoid performing a pelvic examination
presentations have already been defined by the College
of Family Physicians of Canada for finishing family
medicine residents. すでに多くのKFはカナダの家庭医向け
に定義されており、下記にて情報開示されている
• http://www.cfpc.ca and search for “key features”
• Or google “College of Family Physicians of Canada key
features”
内診の実施を回避する
• 3. Avoid discharging home from the emergency department
救急外来から帰宅させない
• 4. Order a pelvic ultrasound
骨盤内超音波を指示する
53
54
Developing cases and questions
Writing the Case Scenario シナリオ記載
• Problem Selection and Context Definition
• Usual elements: 通常の要素
• Patient’s age and gender, presenting site (office,
emergency room etc.), patients presenting condition,
followed by whatever clinical details are required. 患者
の年齢性別、場（診療所・救急など）、患者の症状など必要
に応じて様々な情報
• Think of real patients you have seen in practice.
実際に診療している患者を想起する
• Avoid “medicalese” – e.g. say “coughed up blood”
rather than “hemoptysis”
• Key Features definition
• Case scenario シナリオ
• Test questions and format
• Scoring keys
• Pilot test the question.
医学用語を避ける：血痰ではなく、咳をして痰に血が混じっ
たという表現
9
2013/12/12
55
56
• At 20:00h, the nurse in the emergency department asks you to
see a 24 year old woman who has two children and is now 31
weeks pregnant, reporting of bright red bleeding coming from
her vagina that began about two hours ago. The three sanitary
napkins that she used were completely soaked. Her pregnancy
has been normal, as were the two previous ones. She has not
had any pain in her belly and she says that the fetus is moving
as usual.
夜8時ERに妊娠31週の24歳女性が2時間前から鮮血の不正出血
で来院。ナプキンを3回交換。妊娠は3回目でいずれも順調。腹痛な
し。胎動正常。
• Her BP is 110/70 and her pulse is 92/min. Your exam of the
abdomen reveals a uterine height of 31 cms with soft, non
tender uterus. The fetus is in a breech position and has a heart
rate of 150/min. No bleeding has occurred since she arrived
25 minutes ago.
BP110/70, PR92. 子宮底長31cmで圧痛なく軟。胎児は骨盤位で
150bpm。25分前に来院した後は出血は見られていない。
Developing cases and questions
• Problem Selection and Context Definition
• Key Features definition
• Case scenario
• Test questions and format
テスト問題とそのフォーマットの設定
• Scoring keys
• Pilot test the question.
57
58
Test Questions and Format
Test Questions and Format
• Two formats recommended:2つの推奨フォーマット
• Short answer “write in”. 短答式で「記入」
• Short menu responses リストを挙げる
• Use a direct formulation: 直接的な明快な言い回しで
• E.g. What is your leading diagnosis at this time?
例：この時点での主たる診断は何か
• NOT: Your leading diagnosis is _____________.
悪い例：主たる診断は（
）である
• Present list in alphabetical order ABC順に並べる
• May include correct but not key options – not scored
• Questions must derive from the key features you have
defined. 問題は定義したKFに基づかなければならない
• Questions must focus on clinical actions or decisions
問題は臨床決断に焦点を当てたものにしなければならない
• Not on the reasoning behind these actions
決断の背景にある考え方に焦点を当てるのではない
間違いではないがキーでもない選択肢もあり得る（スコアには含めない。
例：安定狭心症患者に食物繊維の多い食事を指導する）
• To develop the menu, can pilot test a question as a write in question,
and use the incorrect responses to make the list.
リストを作るため短答記入問題で試用し、誤答を列挙することも
• Only one action per option.
各選択肢は単一のアクションを入れる
• Almost always, include “none” as the last option.
「いずれでもない」というオプションはほぼ常に入れておく
59
60
Test Questions and Format
Test Questions and Format
• In general, use write in questions for questions that test
• Every question is followed by one of the following
diagnoses, management and prescriptions.
一般的に記入式は診断、マネジメント、処方の問題に利用
• Use short menu questions for history and physical exam
findings or investigations.
リストは病歴、身体診察、検査に利用
• Use standard questions as much as possible.
出来る限り問題は標準化
statements:
全ての問題は以下のいずれかの形式をとる
• List (select) only one
１つだけ選べ
• List (select) up to X
X個まで選べ
• List (select) as many as are appropriate
適切なものをいくつでも列挙せよ
• E.g. What is your leading diagnosis at this point?
例：この時点での主診断は何ですか
• What tests, if any, will you order next?
次に検査をするとすればどれを指示したいですか
10
2013/12/12
61
62
Question 2. What steps will you take in your immediate assessment
and management of this patient? Select as many as are appropriate.
問題2：この患者の評価やマネジメントに必要なものをいくつでも選べ
• Question 1: What is your leading diagnosis at this time?
List only one, or write “normal” if you judge this situation
to be within normal limits.
問題１：この時点での主診断は何か。解答は１つ。正常範囲
内と考えたなら、「正常」と記載しなさい。
• 1.________________________________
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
Artificial rupture of members
Cervical swab for chlamydia
Complete blood count
CT scan of abdomen and pelvis
Cross and match for 2 units of blood
Discharge home with instructions to
return if bleeding recurs.
Fetal scalp monitor to assess status
of fetus
HIV serology
INR
Large bore IV line
Pelvic ultrasound
Pelvic examination
Syphilis serology
Ultrasound using vaginal probe
Vaginal swab for Group
B streptococcus
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
人工破膜
子宮頸部クラミジア検査
全血球血算
腹部・骨盤CT
輸血2単位クロスマッチ
再出血時再診指示で帰宅
胎児頭皮電極モニター
HIV検査
PT-INR
大口径静脈ライン
骨盤エコー
内診
梅毒検査
経膣エコー
膣部B群溶連菌検査
63
64
Developing cases and questions
Scoring key
• Problem Selection and Context Definition
• List of the correct responses, and a system of assigning
• Key Features definition
numerical scores to the keyed responses.
正答のリストと、点数を付けるためのシステム
• All the correct answers appearing in the scoring key
should be part of the key features.
キーのスコアに表される正答は全てKFの一部であるべき
• Scoring should also be discussed by a group of clinicians
– e.g., how many not harmful tests are too many?
臨床家のグループでスコアは議論を要する。
例：リスクのない検査をどのくらい多くやったら多すぎるのか
• Case scenario
• Test questions and format
• Scoring keys keyの点数設定
• Pilot test the question.
65
Scoring key – Key feature 1:
Given a pregnant women experiencing third trimester vaginal
bleeding with no abdominal pain, the candidate will consider
placenta previa as a leading diagnosis.
KF1：妊娠後期に腹痛のない出血に対し主診断に前置胎盤を想定
66
Key feature 2. Given a pregnant woman with third trimester
vaginal bleeding with no abdominal pain, the candidate will avoid
performing a pelvic examination because it may cause fatal
bleeding. KF2：出血を引き起こす可能性がある内診を避ける
Score
Criteria クライテリア
Score
Criteria クライテリア
Synonyms 同義語
1.0
1.0
Placenta previa
前置胎盤
Marginal placenta, low
placenta, low insertion
辺縁前置胎盤、低置胎盤
Did NOT select #13, pelvic examination
「13. 内診」を選ばない
0
0
Listing more than one
response
2つ以上解答
-Artificial rupture of membranes
-Cervical swab for chlamydia
-Fetal scalp monitor to assess status of fetus
-Pelvic examination
- Ultrasound using vaginal probe OR
- Selecting more than 6 items
人工破膜、頸部クラミジア検査、頭皮モニター、内診、
経膣エコー、6項目以上を選んだ場合
11
2013/12/12
67
Key Feature 3: Given a pregnant woman with third trimester
vaginal bleeding with no abdominal pain, the candidate will avoid
discharging home from the emergency department.
KF3：ERから帰宅させないようにする
68
Key Feature 4: Given a pregnant woman with third
trimester vaginal bleeding with no abdominal pain the
candidate will order a pelvic ultrasound to confirm the
placenta location.
KF4：胎盤の位置確認のために骨盤エコーを依頼
Score
Criteria クライテリア
Score
Criteria クライテリア
1
Did not select “Discharge home with instructions
to come back if bleeding starts again”
「再出血時再診指示で帰宅」を選ばない
1.0
Orders pelvic ultrasound
骨盤エコーを依頼
0
0
Discharge home with instructions to come back
if bleeding starts again OR
Selecting more than 6 items.
「再出血時再診指示で帰宅」、6項目以上を選択
Ultrasound using vaginal probe or selecting more
than six items.
経膣エコー、あるいは6項目以上を選択
69
• Note:
• 3 points for the one question – “What steps will you take
next?” 「次のステップは」は、1問で3点
• Each point related to the key feature.
点数はそれぞれがKFと関連づけられる
（問題2にKF2、KF3、KF4が含まれている）
70
Key feature
Question
number
問題番号
Points
Diagnosis of placenta previa
前置胎盤の診断
1
1
Does not do a pelvic exam
内診をしない
2
1
Does not send the patient home 2
帰宅させない
1
Orders a pelvic ultrasound
骨盤エコーを依頼
1
2
点数
71
72
Resources for writing Key Feature
Problems KF問題の情報源
Biggest challenges
• http://mcc.ca/about/test-committee-resources/
• Time to write the questions:
• Get together groups of experienced clinicians, train
them in how to write and score such questions
• Time to score the exam:
• Write in questions are scored manually up until now by
MCC.
• Research in progress on computer scoring of the write in
questions
12
2013/12/12
73
• Using it on a smaller scale: 小規模に試用した研究
• Study by Hatala and Norman
• Internal Medicine Clerkship 内科クラークシップ
• 8 week clerkship, therefore 6 exams 8週で試験6回
• Blueprint taken from internal medicine clerkship
objectives 内科クラークシップ学習目標からブループリント
• 5 faculty members assigned to develop and review key
features for 14 to 18 clinical problems each over an 8
week period – total 82 problems 5人の教員が、8週間で
毎回14~18の臨床問題を作成、合計82個の問題を開発し、
レビューした
74
• Workshop of faculty and residents to write the
questions 教員と研修医が作問のWSを受けた
• They were given the problem, the age of the patient, the
clinical situation and the defined key features
問題、患者の年齢、臨床状況、定義されたKFを与えられた
• Generated 35 problems. 35問を作成
• Remainder (47) done by one faculty member 残りは1名
が作成
• 15 problems per exam (2 hours) 2時間で15問
• Cronbach’s alpha 0.49
75
76
Conclusions:
References:
• An excellent way to test clinical decision making
• Bordage G, Meguerditchian A, Tamblyn R, Practice Indicators
臨床推論をテストするために優れた方法
• Consider: 是非こんな形で考えてみましょう
•
• Start small - clerkship exam 小規模からスタート
• Then graduation exam 次いで卒業試験に
•
• Eventually on the national exam? 最終的には国家試験？
• Predictive validity in the Japanese context should be
studied. 是非日本での妥当性研究をすべき
• We need to determine what it would add to the current
test methods. 現状の評価方法にKF問題が何を加えるかを
決める必要がある
•
•
of Suboptimal Care and Avoidable Adverse Events: A content
analysis of a National Qualifying Examination. Acad. Med
2013; 88: 1493-1498
Hatala R, Norman G, Adapting the Key Features Examination
for a clinical clerkship. Medical Education 2002;36:160-165
Page G, Bordage G, The Medical Council of Canada’s Key
Features Project: A More Valid Written Examination of Clinical
Decision-making Skills. Acad. Med. 1995;70:104-110.
Page, G, Bordage G, Allen T, Developing Key-feature Problems
and Examinations to Assess Clinical Decision-making
Skills.Acad.Med 1995;70:194-201
Bordage G, Brailovsky C, Carretier H, Page G, Content
Validation of Key Features on a National Examination of
Clinical Decision-making Skills. Acad.Med 1995;70:276-281.
77
References
• Medical Council of Canada, Guidelines for the Development of
Key Feature Problems and Test Cases. Available on-line at
www.mccc.ca
• Tamblyn R, Abrahamowic M, Dauphinee D, et. al. Physician
Scores on a National Clinical Skills Examination as Predictors
of Complaints to Medical Regulatory Authorities. JAMA 2007;
298(9):993-1001
13