Ever since Typesafe’s Jev came out, people have been raving about its “one-shot” classification capability. We decided to put our news archives through Qwen (a “regular” llm) and Jev (a classifier) to bucket the sentiment expressed in each of them into Positive, Neutral and Negative.
Is Jev lazy? Picking “Neutral” is the path of least resistance, isn’t it? What if we split it by media type?
It looks like Jev loves being neutral. Either Indian financial media is mostly unbiased or Jev is not as “one-shot” as claimed to be. Thanks to Jev’s neutrality, the two models agree only 43% of the time.
Given their differences, are they useful?
There is no evidence that extreme media sentiment reliably predicts subsequent index direction. The positive post-event paths may simply reflect the general upward drift of the indices.
It appears that sentiment is in the eye of the analyzer and largely useless in predicting low frequency market direction. Details about the methodology and results are here.
Code and charts are on github.