Optimal Card Hand Size

I would say 3-8 is a range you will probably fall within. If you wish to read the rest of my banter, please do. Otherwise, 3-8 cards with 5 cards being the nice poker-sized hand with plenty of options would be my starting amount.

Considering you are not playing Werewolf with a single role card and no one wants the mega-ultra-colossal loser hand you can get playing UNO, I think you need to consider what are the mechanics involved in the game besides the cards and what amount of choice does a player have in using the cards. Do they need to hold many cards even though the choices are few? Does each card do more than one thing? 3 cards that can be used individually 3 different ways represents a possible 9 choices--if they can be used in combination, then the number of choices increases signifcantly. You said this is a "board game that utilizes cards" rather than a "card game." Let's take a couple examples and explore a bit...

In the simple card game Guillotine, a player starts with 5 action cards and along the way can end up with more or fewer depending on how the cards are played...no limit either way, but you always pick another action card as part of your turn. These action cards are single function, but they are all different types of effects.

Contrastingly, in the board game Founding Fathers, you only have a hand of 3 usually, except for certain situations that give you 4 or very rarely 5. These cards are multi-functional (you can use them to vote, debate, enact events, or be discarded for new cards). You always draw up to your max hand of 3 (or 4 if you have John Langdon's persistent event).

Pandemic and Castle Panic are cooperative board games that have maximums of 7 and 5, respectively. Then you must begin wasting cards to the discard pile.

In Succession, there are 3 different types of cards, and each player holds 4 cards to start. You can gain more along the way, I believe.

In Ticket to Ride, there is no limit to the number of train cards (right?) a player can have. You can select 2 "regular" cards or 1 visible wild card. However, you can play sets of cards which can be up to 6 at one time.

My point with all of these examples is that I don't think there is a formula. I think there is a question as to whether the number of cards along with the other game mechanics provides the right amount of options so that the player has a tough choice to make. For example, if the cards can be used individually for combat OR used in combination for bidding or payment, it may be that a maximum of 4, 5 or 6 forces some tough decisions while providing many ways to play them. However, with a mechanism like set collection, it may be that you don't want to limit as much.

If there is a thematic link, that may influence this decision as well. For example, if the theme was about the 4 elements or 4 food groups...that could be your limiter. If you were playing 1 card per day in the game, then you might be limited to 5 (weekdays),6 (with a day of rest), or 7 (full week). You get the idea here, I'm sure.

If you consider strictly physical functionality, I think anything in the range of 3-8 seems reasonable. However, I also think those other factors I discussed above will be more influential in my opinion.

Hope this helps. Good luck!