I like the dice badges, and I appreciate their minimalistic design: they're dice if you look (the RPG.SE banner d20 is right there for the comparison) but they don't call attention to themselves. I think it's totally fine that they aren't obviously RPG-related symbols. It's not like english.se's badges are iconic of their topic, and they look great!
Still, making the badges more immediately recognisable as dice could be nice if they don't get too flash in the process. The trick is, badges are an ubiquitous part of the site design and shouldn't be overly complex or they clutter up every page they're on. Maybe a d6 with transparent pixels for the pips, similar to movies.se's film-reel badge?
I'm firmly against a more "fantasy-like" design for the badges, though. Polyhedral dice are universally recognised in the RPG community as emblematic of our hobby; fantasy visuals represent only a portion of the hobby's subjects while also evoking other mediums which use those visuals too. A sword or dagger or staff doesn't say "role-playing game" to me, any more than a laser rifle or a shoggoth does. (I feel the scifi Stack's choice to draw on a single franchise for its badge design is a little tone-deaf, actually.)