My HP headcanons all have and will fit squarely and comfortably within the source text descriptions but that’s just my own preference for imagining characters, not a rigorous standard I’d hold other people to.
I suspect that Jo Rowling probably imagined James and Harry as white too, i don’t mind that, that’s her business.
Personally I’m mentally acting against the white-as-default-unless-otherwise-specified that’s pervasive in imagining media especially in predominately white fandom culture.
I’ve generally operated on a POC-unless-otherwise-noted basis with how I imagine characters. My HP headcanons aren’t What If Everybody In Harry Potter Was Black Instead Of White. They only read that way if you’re still stuck in the white-as-default zone.
James’ skin color, hair texture, etc to my knowledge has never been specified and that means to me that even a reader who strictly contains themselves within the bounds of canon is free to imagine him as any number of ethnicities.
In absence of a specified race I chose to imagine the one that makes the story most compelling to me.
My James is black because that creates the most personally compelling racial background for my Harry. It is informed by my interpretation of the canon interactions between the Potters and the Evans/Dursleys, whom Jo Rowling and I probably imagine very similarly. It is informed by my experience as the black mixed-race child of a black man and a white woman who grew up more or less estranged from both my parents largely in the care of my white maternal aunt and her family, household, values, and prejudices. It is informed by my personal desire for a black mixed-race hero story.
Probably most important to me, my desire to create and disseminate content that involves non-white interpretations of popularly-imagined-white-by-default characters reflects my desire to speak to people like me, who are not used to seeing faces like theirs represented in the popular media they consume. It reflects my fight against white-as-default. It reflects my desire to contribute to young people of color feeling empowered by popular fiction and not othered by it. It reflects my desire not to let blockbuster casting directors dictate what you may or may not imagine the characters that populate your fiction to be.
It’s not arbitrary and it doesn’t come from nowhere, but if it did, that would be fine too. All my interpretations are based squarely in canon. But if they weren’t, that would be damn well acceptable. Squeeze representation out of anywhere you can feel it and fabricate the rest. Own your fiction.