Construction is known as one of the most stressful occupations due to its involvement with physiologically and psychologically demanding tasks performed in a hazardous work environment. Because workers' stress is a critical factor that adversely affects workers' productivity, safety, well-being, and work quality, understanding workers' stress should take precedence in the management of excessive stress. Various instruments for subjective measurement towards one's perceived stress have been used, but such methods rely on imprecise memory and reconstruction of feelings in the past. Recent advancements in wearable Electroencephalography (EEG) devices possess a potential for quantitative measurement of human stress by directly capturing central nervous system activities from stress. However, its capability of measuring field workers' stress under real occupational stressors remains questionable. This research thus proposes a framework to recognize construction workers' stress at the field based on their brain activity recorded from a wearable EEG. Specifically, this framework applies a supervised learning algorithm-support vector machine-in detecting workers' stress while working in different conditions. Workers salivary cortisol levels, which is a stress-related hormone, were used to label the tasks as low or high-stress level. Relevant time and frequency domain features in EEG signals were calculated. Results yielded a high of 71.1% accuracy using SVM in recognizing workers' stress, which is a very promising result given that stress recognition with an exquisite EEG device in the clinical domain has at most the similar level of accuracy. The results show the potential for recognizing construction workers' stress at the field by applying machine learning algorithms using workers' brain waves recorded from a wearable EEG device. This EEG based stress detection approach will help to enhance workplace environment and conditions as well as to improve workers' health by early detection and mitigation of the factors that cause stress.