Keywords

1 Introduction

Timber is a renewable construction material with a low carbon footprint. It holds significance in being both a sustainable modern-day building material, as well as a traditional material that intertwines with cultures all over the world. Though there are a range of positive environmental and engineering properties associated with timber, as natural material, it contains a high level of variation in material properties as the result of uneven natural growth and non-uniform environmental conditions. Defects such as knots, checks, splits, and wane are common [1], but their presence can only usually be identified and assessed after the fabrication of the sawn board. In Australia, sawmill reports by Harding have found up to 57.5% of sawn board can fail structural grading requirements [2]. As compared to structural timber, out-of-grade timber is far less desirable for most construction or manufacturing operations, and consequently it has a low market demand and is often woodchipped and/or sold at a loss [3].

A common method in the timber industry for value-recovery in out-of-grade timbers is through the cutting out of defective parts, and the subsequent joining of the non-defective (clear wood) segments into a full-length structural board (Fig. 1). Joining is normally achieved with finger joints, as they have adequate strength for typical structural timber applications, such as wood trusses and laminated beams [4]. Though the pipeline is mostly automated, it still often requires humans to visually identify the defect and mark them out for removal. This removal process is largely dependent upon the experience of the worker and is also the time-limiting operation in the material-recovery process. The segment jointing method itself is also is a source of leftover clear wood material, as pieces are jointed to a length based on sequential segments, and trimmed down to a pre-determined final length.

Fig. 1.
figure 1

Typical sawmilling process: a) The tree is harvested and docked into transportable lengths (approx. 6 m) with varying branch distribution according to: silvicultural practice, climate, soil, planting density etc.; b) each log is sawn into usable framing members and is subject to the naturally occurring structural defects; c) sawn members are sorted into those with and without defects; d) members free from defects are certified for use in structural applications; e) members with defects are used in non-structural application or woodchipped; f) defects are removed, resulting in short lengths, which provide: g) an inventory of short, unique lengths [3].

Full size image

This research investigates the possibility of enhancing this process with machine learning based image recognition and decision-making algorithms. Image recognition is widely adapted within different industries, such as the classification of packages for delivery [5]. The advantage of using image recognition and machine vision is that it can be achieved with minimum hardware investment cost, often simply by adding a camera to an existing adaptive industrial pipeline [5]. The images of timber collected from a machine vision manufacturing process can also feed back into the image recognition and defect detection system, allowing its accuracy improve over time with expanded training datasets, and allowing the system to self-improve and or evolve with changing timber stock. Additionally, introducing a machine vision system allows a live data record to be collected for each clear wood segment generated. Such information provides a potential to introduce a decision-making algorithm that can optimize the joining process to reduce left over material, or for customization of components with different target lengths.

2 Defect Recognition and Removal

2.1 Pre-process the Image for Segmentation

The images of timber used in this research study are collected from an inline Machine Stress Rating (MSR) system model 720 HCLT. For any testing timber board, it collects a series of images and also directly measures the board apparent Modulus of Elasticity (E), with the two datasets then used to assign a final board grade. For the visualization requirements of the present project, the collected image required prepossessing by decreasing the noise, removing distortion, and removing the background:

  • Image de-noise: Spikes of light noise at the border of the timber is widely seen in the scan images, owing to the rapid-rolling scanner used in the industrial setting. This noise can be significantly reduced by reading the horizontal proximity of each pixel to evaluate if it’s a ‘spike region’, which is characterized by a small area of foreground (board) pixel, bordered on both sides by background pixel (Fig. 2a–b).

    Fig. 2.
    figure 2

    (a). Raw image collected from MSR; (b). Image after denoise; (c). Image after un-distortion and background removal

    Full size image

  • Un-distortion and background removal: the central axis of the scanned timber member is often slightly un-even or non-linear. A pixel column line-scan process aligns the foreground pixels in the denoised image to a linear axis, and removes any background noise. The final image is a consistent and formatted visualization of board material and defects (Fig. 2c).

Currently, this pre-processing workflow is developed based on image collected from scans from the machine grading system. When compared to images collected from the 7200 HCLT, images collected from typical camera hardware would contain far more noise, an uneven background, and uneven lighting quality. When connecting with an industrial manufacture environment, though following similar principles, the system can be further calibrated based on the image collected by the camera feed, accommodating the local lighting and environmental conditions.

2.2 Preparation of the Classifier

Based on the collected image, a classifier is developed to identify the location of defects. This classifier is trained to assign a discrete categorized label for each pixel in the timber scan image, to identify if the pixel belongs to the following three classes: defective timber, non-defective timber, or background (non-timber). In addition, the classifier also outputs a probability map that describes the probability of each of the three classes at each pixel.

The collected image is interactively labelled by defining the region of “Background”, “Defective Timber”, and “Non-Defective Timber” in the Fuji distribution of Weka. The training of the classifier took place using the API of the Trainable Weka Segmentation [6]. The classifier is trained based on a FastRandomForest model, which is an enhanced multi-tread version of the ensemble learning method for classification, regression, etc., developed by Fran Supek [7]. The model has a batch size of 100, 8 thread, 200 trees. The training feature includes Gaussian Blur, Sobel Filter, Hessian, Difference of Gaussians, and Membrane Projections. The features are extracted, converted and formatted to a set of vectors for the Weka classifier. The features are calculated with 8 threads after the labelling of training data in Weka.

2.3 Preparation of the Classifier

After the training of the classifier in the intuitive Trainable Weka Segmentation GUI, the classifier is applied in the backend of a simulation interface. It takes in an image of the timber piece to generate classification and probability maps for the image. The classification map returns the classification of each individual pixel in one of three classes: “Background”, “Defective Timber”, and “Non-Defective Timber” (Fig. 3c–e). The classification map set includes a probability map for each of the classes, where each pixel is assigned a 0–1 value that describes the probability of the selected pixel being in given class (Fig. 3a–b). In this research, the raw probability value is used to determine the defectiveness of each section of the timber. The classifiers can further be trained to increase accuracy, by collecting additional labelled images in the process.

Fig. 3.
figure 3

(a) Raw image collected from MSR; (b) Binary classification map; (c) Probability map for non-defective timber; (d) Probability map for defective timber; (e) Probability map for background.

Full size image

With the probability map for defective timber, the image is further processed to simulate the removal of timber defects. There are two parameters used to define the threshold of identifying a timber defect, giving a user the option to calibrate the range and sensitivity threshold for defect removal. The first parameter defines the threshold of probability for pixels to be considered as defective, which is currently set at 75%; the second parameter is the percentage of defect pixel in a 1 pixel-wide column slice, which is currently set at 5%. Exceeding which, the corresponding slice of timber will be considered as defective area.

The defect removal process will scan through the processed timber image and remove any defect areas composed of more than 5 consecutive defective slices, which translates into a defect with horizontal dimension of 3 cm or greater. For the remaining material, graded timber segments that are 20 cm or longer are stored in a dynamic stock of clear wood timber segments (Fig. 4).

Fig. 4.
figure 4

(a) Raw image collected from MSR; (b) Probability map for defective timber; (c) Non-defect timber segments; (d). Jointed piece of standard length.

Full size image

3 Decision Making for Joining Timber Segments

Each iteration of inputting a new piece of timber will add an indefinite number of segments of non-uniform lengths into a ‘dynamic stockpile’ of segments, ready for jointing. Owing to manufacturing standardisation, there is a uniform set target length for a final jointed timber piece, taken as the common industry length of 4800 mm for present study. However, into jointing up to the target length, a new challenge is encountered in the selection of timber segments from the dynamic stock pile: how to select segments to exceed the target length, but with a minimum of overrun so as to minimise the waste generated from final trimming.

A new system is proposed to resolve this challenge. The system criteria for deciding which pieces of timber segments should be joined together follows premises as summarised below:

  • Generate different length combinations from a selection of a set of timber pieces that provide the user’s target length (4800 mm) and a small overrun allowance (target total length +200 mm).

  • Of all combinations, those with the closest proximity to the target length are prioritized for joining.

  • On receiving a new input of segments, the new segments will join the left over segments int the dynamic stock pile waiting to be jointed. The dynamic stock is re-scanned to update the possible combinations and joining priority that meets the previous two criteria.

  • Monitor the amount of wastage that comes from the difference between the user’s target length and the actual total length of the selected combination. This wastage may be the result of the total length being longer than the user’s target length.

The most extensive algorithm to explore the subspace of potential combination of segments is the Exhaustive recursion method, where all possible solutions are explored and compared. However, the computational cost for such method explodes exponentially as the pieces in the stockpile increase, causing an infeasibly-long processing time.

Given the restriction in computational power and processing time, this study instead implements a backtracking logic to select the appropriate combination of timber segments, which is far less computationally demanding, but similarly effective to the exhausted recursion method. Figure 5 provides a breakdown of the jointed outcome, from receiving the first 10 randomly selected out-of-grade timber piece as input to the system sequentially.

Fig. 5.
figure 5

Stock condition with 10 under graded timber pieces as input sequentially.

Full size image

The pseudocode below represents the backtracking selection logic behind the scene:

figure a

After processing a randomly selected 20 pieces of defective timber, the system manufactures 14 pieces of 4,800 mm long full length structurally finger joined timber without defect, with a total length of 67,200 ‬mm. At same time, this system generates only 58 mm of total waste leaves 6 pieces in the dynamic stock pile, 4,308 mm in total length. This proves the efficiency of the proposed system in salvaging non-defective timber, with a utilisation rate of over 99.96%. The track of inventory change is as below in Fig. 6:

Fig. 6.
figure 6

(a). Change in dynamic stoke pile; b). Changes in the accumulated waste; c) Changes in the number of completed pieces.

Full size image

4 User Interface

In this project, a platform is developed to simulate the backend process of image recognition and decision-making to evaluate the feasibility of the proposal and its efficiency. By calling to get next piece of timber, the system will pop in the scan image of the next timber from the database to simulate the collected data from camera in a timber mill. This image is been processed with 4 steps to recognize and remove defective area. Each of the 4 steps including image denoise, background removal with un-distortion, probability calculation, and defect removal is visualized sequentially in the left column for the user to understand and evaluate the system.

The segments without defect are stored in a dynamic stock. The next column provides the visualization of the longest 5 segments in the dynamic stock, waiting to be jointed. At each input of new segments, the decision-making algorithm evaluates the dynamic stock of segments, and picks the appropriate segments to joint into a full piece with target length. The latest 5 completed pieces are displayed at the right column, along with access to the folder keeping the entire stock of completed jointed timber pieces (Fig. 7).

Fig. 7.
figure 7

Graphic interface of the system

Full size image

In the interface backend, the program tracks the percentage of wasted timber, salvaged timber, and the distribution of segments for the completed timber. Thus, efficiency of the system is tracked throughout the process. Also, the target length can be changed at any time to test the robustness of the system.

The pre-training of the image classifier is performed within the interface of Fuji distribution of ImageJ. It enables the interactive tracing for different classes of elements that can be performed by worker without any computational background, thus offering an intuitive interface to translate the experience and judgement of worker into the classifier that provides a similar process as a visual grading process.

5 Discussion and Future Development

The system developed in this research provided the simulation of the manufacturing of structural finger jointed. This simulation system is a reflection on an established industrial manufacture system, and a further investigation the potential of applying machine learning and decision-making algorithm to improve its efficiency in terms of increased processing speed and reduced material waste. Such development can be potentially achieved with very low-cost hardware investment, by simply adding an additional camera into an processing line, for capture of material properties in real-time.

The effectiveness of the system arises from visually-identifying the variation in material property and making intelligent, algorithm-driven decisions for material processing based on the property of individual material. The current training data is based on a manual labelling process similar to the industry. Reflecting on the workflow, there is potential to further improve the accuracy and efficiency of the system by using the MOE data currently collected from machine stress grading processes.

Compared to the traditional manual process of labelling and categorizing, the proposed system is able to systematize the decision-making process, quantify its output, and provide a consistent evaluable structure for the workflow. By embracing a digitized process for machine learning based defect detection, accuracy could improve overtime with the increasing real time collection of datasets, contributing to more accurate removal of defects. With future development, this feature of real-time updating would pave the way for a self-evolving dynamic classifier. A defect detection classifier can adaptively transfer between different scenarios, while the manual labelling process relies on the non-transferrable experience of individual. With the digital data for each piece collected during the manufacture process, a live inventory-based optimization for the dynamic stock becomes possible. This increases the theoretical material efficiency to as much as 99.9%, and provides possible avenues for future adaption of the system to provide customizable parts of different targeted lengths.

6 Conclusion

This research discussed the possibility of using machine vision and decision-making algorithm to optimize a standard industrial workflow in salvaging defective timbers. Such system work with the real-time processing of material data in a pipeline, instead of a static database. It allows for a versatile adaption into the existing industrial workflow with minimum cost.

By introducing this digital and intelligent workflow, the system gains additional potential to increase in material efficient and environmental friendliness by significantly reducing the material wasted during the manufacture process. Also, embracing the intelligence in the system, it pave the way for future upgrade of the production system for optimized mass customization of non-uniform members.