Hello! Your answer is no, prokaryotes do not have a cellulose wall.
Prokaryotes do have a cell wall, however, they are made of other things. There are two kingdoms of prokaryotes - eubacteria and archaebacteria. Eubacteria have cell walls with peptidoglycan, and archaebacteria have cell walls made of polysaccharides and proteins.
Hope this helps!