Truace
Subscribe

Deep forest

Current deep-learning models are mostly built upon neural networks, i.e. multiple layers of parameterized differentiable non-linear modules that can be trained by backpropagation. In this paper, we explore the possibility of building deep models based on non-differentiable modules such as decision trees. After a discussion about the mystery behind deep neural networks, particularly by contrasting them with shallow neural networks and traditional machine-learning techniques such as decision trees and boosting mac…

By the Truvace desk1 min read5 sourcesPeer-reviewedTRV-2026-0205· cite & verify
Deep forest
Skipping Cow timber sale, Tongass National Forest : final EIS environmental impact statement and record of decision by United States. Forest Service. Alaska Region Wrangell Ranger District (Alaska). Public domain

In brief

As of the 2018-10-08 publication, researchers proposed deep forest, a decision-tree ensemble that stacks non-differentiable modules in layers to replicate deep learning characteristics without using backpropagation for training.

The approach matters because it suggests deep learning can be achieved outside differentiable neural networks, potentially lowering hyper-parameter tuning burden; uncertainty remains about how broadly the reported robustness generalizes beyond the experiments described and whether theoretical conjecture is fully validated.

Main points

  1. Proposes deep models built on non-differentiable modules such as decision trees instead of parameterized differentiable non-linear modules trained by backpropagation
  2. Conjectures deep neural network success owes to layer-by-layer processing, in-model feature transformation and sufficient model complexity
  3. Deep forest is a decision-tree ensemble with fewer hyper-parameters than deep neural networks and data-dependent complexity determination

The gain

Decision-tree ensemble approach called deep forest achieves excellent performance across different domains using the same default setting while requiring fewer hyper-parameters and automatically determining model complexity without gradient-based adjustment or backpropagation.

The rundown

The authors contrast deep neural networks with shallow networks and traditional techniques such as decision trees and boosting machines to motivate three characteristics for success.

Experiments reported as of 2018-10-08 indicate robustness to hyper-parameter settings, allowing the same default configuration to work across data from different domains.

Sources

  1. Peer-reviewedWater Resources Research2018-08-30
  2. Peer-reviewedJournal of Network and Computer Applications2021-07-09
  3. Peer-reviewedNational Science Review2018-10-08
  4. Peer-reviewedInternational Journal of Mining Science and Technology2021-09-13
  5. Peer-reviewedSystems2024-01-24

The debate