I have recently done an ‘experiment’ with ChatGPT, and have discovered that while it is an impressive tool, it may not be the best fit for academic research. Through a series of inquiries on fundamental osteoporosis knowledge, I have noticed that:
1. ChatGPT can provide some credible answers
When I asked “Can you tell me what is osteoporosis”, ChatGPT gave a fairly accurate but not very in-depth answer. ChatGPT responded, “Osteoporosis is a medical condition characterized by weak and brittle bones, leading to a higher risk of fractures.” It did not mention the structural aspect of bones, which is a shortcoming.
When asked about the efficacy of denosubmab in treating osteoporosis, I found the answer to be fairly good. Although not complete, the basic information about the drug is good enough for beginners or those seeking to learn more about osteoporosis.
When asked whether vitamin D supplementation is effective in reducing fracture risk (“Does vitamin D supplement reduce fracture risk?”), ChatGPT responded quite well. The good thing is that the last part of ChatGPT’s answer recommended asking a healthcare professional for personalized advice.
2. ChatGPT can understand Vietnamese to some extent
I then switched to Vietnamese “Loãng xương là gì?” (What is osteoporosis?), ChatGPT responded, “Loãng xương là một tình trạng y khoa, được định nghĩa bởi xương yếu và dễ vỡ, dẫn đến nguy cơ gãy xương cao hơn.” This answer is actually a translation of the English version above. However, the translation is somewhat awkward (“tình trạng y khoa”) and the sentence is not clear and complete (“nguy cơ gãy xương cao hơn”).
When I asked about the risk factors for bone fracture in English (“What are risk factors for bone fracture?”), ChatGPT listed comprehensive risk factors. When I asked in Vietnamese (“Yếu tố nguy cơ loãng xương là gì?” — What are the risk factors for osteoporosis?), the answer, once again, was a translation of the English version. However, the answer is relatively complete and accurate.
3. ChatGPT can provide examples, even specific formulas
I then switched to epidemiology by asking for the formula for calculating NNT (number needed to treat), it provided the correct formula and a very specific explanation. Additionally, when asked about an example of the concept of “collider bias,” it presented an…