No. I promise you.
The bulky look you're picturing comes from years of lifting very heavy in the gym, eating for it on purpose, and usually certain supplements too. That's not what you get from dumbbells at home.
What you get is a toned, strong, defined look. Toned and building muscle are the same path. Muscle is what creates the toned look, and it raises your metabolism so you burn more fat just living your day.
Strength training is just as important for women as it is for men. Keep the dumbbells in. You're on the right track.