Intermittent androgen suppression in the prostate cancer is often relapsed by the increasing of prostate specific antigen level during the on-treatment. Historically, chemotherapy has had a limited role in the treatment of prostate cancer. However, new agents are showing promise in patients with advanced disease. Intermittent androgen suppression plus chemotherapy in pulsed pattern has become an indispensable clinical scheme for prostate cancer, which is presented to describe the transformation mechanism for three kinds of cancer cells in this paper. The model is then extended to include the residual effect of chemotherapy which suppresses the cancer cells production, thereby preventing the relapse. The optimal controls represent the efficiencies of both intermittent androgen suppression and chemotherapy in suppressing relapse of prostate cancer. Based on an optimal algorithm, numerical simulations are implemented not only to show the optimal durations of on- and off-treatment and chemotherapy dosages but also to present the effectiveness of different strategies in inhibiting the relapse for three types of patients. Results reveal that the optimal intermittent androgen suppression scheme with alterable treatment cycles is pivotal for type I and II patients, in part because it can greatly reduce the on-treatment time and degrade the level of prostate specific antigen. Furthermore, optimal hybrid schedule even averts the relapse of prostate cancer for type II and III patients. Finally, comparing the prostate specific antigen under intermittent androgen suppression schedule with residual effect of chemotherapy to one without residual effect of chemotherapy demonstrates the validity of both our model and algorithms in lessening the prostate specific antigen and decreasing the chemotherapy dosages.