No, cockroaches are not asexual, and require a mate to reproduce.
Females lay up to 40 eggs at a time in a small case, called an ootheca, which looks like a purse shaped like a bean, and an adult female cockroach can lay eggs up to four times per year. Take a look at the illustration to see what the ootheca looks like.
Most species of cockroach deposit the egg casing in a safe and secure place, which is preferably warm and dark and secluded. A few species however, notably the German cockroach, instead carry the ootheca on their body until the eggs are ready to hatch.
It takes about six weeks for the eggs to develop and hatch into nymphs, which are essentially baby cockroaches. The nymphs are colored white when they are born, but quickly turn brown when exposed to oxygen.
Some species of cockroach display maternal instinct, and the nymphs remain close to the mother for several days. And other species produce milk in order to nourish their young.