We live in a small village, so the only "backyard" we have is really the playground that is one house over.
Since my oldest is autistic and youngest has had other therapy needs (speech, occupational) it has taken until last year to start to allow them time alone for an hour or so unsupervised with the rule that neither is to come home without the other (except in an emergency) and that both have a watch and are to come home at x time or we will come get them.
Aside from the ankle break last year (November), they had been reliably able to do that at 9 and 7. They are now 8 and almost 10.