Not really true, but I guess it depends on the country.
In the United States at least, burning your own book, flag, or whatever is legally protected free speech. Just as long as you aren’t destroying someone else’s property.
Context also matters. Burning bibles during a religious service is probably a thin line.