How to control osteochondral differentiation of mesenchymal stem cells at a proper stage is a key issue for articular cartilage regeneration. To solve this problem, injectable scaffolds with different chemical functional groups were designed by introducing one equivalent of α-cyclodextrin (α-CD) carboxylate and α-CD phosphate along poly(ethylene glycol)-poly(l-alanine) (PEG-L-PA) block copolymers. Dynamic light scattering, transmission electron microscopy images, and two-dimensional NMR spectra indicated that the PEG-L-PA block copolymers formed inclusion complexes with α-CD derivatives. Aqueous solutions of PEG-L-PA block copolymers (P), α-CD carboxylate/PEG-L-PA block copolymers (PCC), and α-CD phosphate/PEG-L-PA block copolymers (PCP) underwent sol-to-gel transition as the temperature increased. The storage moduli of P, PCC, and PCP gels ranged from 1000 to 1300 Pa at 37 °C. Tonsil-derived mesenchymal stem cells (TMSCs) were incorporated in situ in the gel during thermogelation of P, PCC, and PCP, which became the three-dimensional cell culture systems with different functional groups. After 21 days of incubation of TMSCs in the P, PCC, and PCP systems, the chondrogenic differentiation biomarker of type II collagen significantly increased in the P system, whereas the osteogenic biomarkers of osteocalcin and runt-related transcription factor 2 significantly increased in the PCP system. Both chondrogenic and osteogenic biomarkers were highly expressed in the PCC system. This study proved that thermogelling inclusion complex systems consisting of PEG-L-PA block copolymers and α-CD derivatives could be an excellent injectable matrix for fine-controlling osteochondral differentiation of mesenchymal stem cells.