Baby Boks have only themselves to blame says coach