This AをBと思う is "to think of A as B" or "to regard A as B". Here A corresponds to 沈黙するレーナ and B corresponds to the interrogative 何. か is a question marker. So you can imagine a question like this:
沈黙するレーナを何と思った(の)か?
What did [he] think of silent Rena as?
→ What did [he] think about Rena's silence?
Since it's placed at the beginning of a sentence, it expresses the speaker's (or the narrator's) speculation about the reason for the subsequent part. See this for other examples.
沈黙するレーナを何と思ったか、カールシュタは気遣わしげに首を傾げる。
(literally) What did he think about レーナ's silence; カールシュタ tilted his head in a caring way.
→ Looking at レーナ's silence, カールシュタ thought of something and tilted his head in a caring way.
(If the narrative part is written in the first person's view, see Eddie Kal's translation in the comment section.)