To my modern tongue, the phrase "I fear" here seemed a little mocking.

I initially understood it along the lines of Alexander Pope's 'little learning' quote from 1709:

A little learning is a dangerous thing; drink deep, or taste not the Pierian spring: there shallow draughts intoxicate the brain, and drinking largely sobers us again.

Yet my modern reading was rather off from what was originally intended:

The phrase was in origin a dismissal of eclecticism, i.e. the "fear" is of the formidable intellectual opponent who has dedicated himself to and become a master in a single chosen discipline. In this first sense ... "one book" taken to mean the Bible.

I much prefer this compared to my default take, which is tainted with the mentality of abundance we of modern times enjoy.

I now read the quote more along the lines of 無心 - which I think is an admirable way to portray one's devotion to learning a book deeply. (Even to the extent of inspiring terror for it.)