Layer-Wise Relevance Propagation: An Overview

Abstract

For a machine learning model to generalize well, one needs to ensure that its decisions are supported by meaningful patterns in the input data. A prerequisite is however for the model to be able to explain itself, e.g. by highlighting which input features it uses to support its prediction. Layer-wise Relevance Propagation (LRP) is a technique that brings such explainability and scales to potentially highly complex deep neural networks. It operates by propagating the prediction backward in the neural network, using a set of purposely designed propagation rules. In this chapter, we give a concise introduction to LRP with a discussion of (1) how to implement propagation rules easily and efficiently, (2) how the propagation procedure can be theoretically justified as a ‘deep Taylor decomposition’, (3) how to choose the propagation rules at each layer to deliver high explanation quality, and (4) how LRP can be extended to handle a variety of machine learning scenarios beyond deep neural networks.

Appendices

Appendices

10.A List of Commonly Used LRP Rules

The table below gives a non-exhaustive list of propagation rules that are commonly used for explaining deep neural networks with ReLU nonlinearities. The last column in the table indicates whether the rules can be derived from the deep Taylor decomposition [36] framework.

Name	Formula	Usage	DTD
LRP-0 [7]	$R_j = \sum _k \frac{a_j w_{jk}}{\sum _{0,j} a_j w_{jk}} R_k$	Upper layers
LRP-$\epsilon$ [7]	$R_j = \sum _k \frac{a_j w_{jk}}{\epsilon + \sum _{0,j} a_j w_{jk}} R_k$	Middle layers
LRP-$\gamma$	$R_j = \sum _k \frac{a_j (w_{jk} + \gamma w_{jk}^+)}{\sum _{0,j} a_j (w_{jk} + \gamma w_{jk}^+)} R_k$	Lower layers
LRP-$\alpha \beta$ [7]	$R_j = \sum _k \Big (\alpha \frac{(a_j w_{jk})^+}{\sum _{0,j} (a_j w_{jk})^+} - \beta \frac{(a_j w_{jk})^-}{\sum _{0,j} (a_j w_{jk})^-}\Big ) R_k$	Lower layers	$\times ^{a}$
flat [30]	$R_j = \sum _k \frac{1}{\sum _{j} 1} R_k$	Lower layers	$\times$
$w^2$-rule [36]	$R_i = \sum _j \frac{w_{ij}^2}{\sum _{i} w_{ij}^2} R_j$	First layer ($\mathbb {R}^d$)
$z^\mathcal {B}$-rule [36]	$R_i = \sum _j \frac{x_i w_{ij} - l_i w_{ij}^+ - h_i w_{ij}^- }{\sum _{i} x_i w_{ij} - l_i w_{ij}^+ - h_i w_{ij}^-} R_j$	First layer (pixels)

1. ($^{a}$DTD interpretation only for the case $\alpha =1,\beta =0$.)

Here, we have used the notation $(\cdot )^+ = \max (0,\cdot )$ and $(\cdot )^{-} = \min (0,\cdot )$. For the LRP-$\alpha \beta$ rule, the parameters $\alpha ,\beta$ are subject to the conservation constraint $\alpha = \beta + 1$. For the $z^\mathcal {B}$-rule the parameters $l_i,h_i$ define the box constraints of the input domain ($\forall _i: l_i \le x_i \le h_i$).

10.B Justification of the Relevance Model

We give here a justification similar to [36, 37] that the relevance model $\widehat{R}_k(\varvec{a})$ of Sect. 10.2.3 is suitable when relevance $R_k$ results from applying LRP-0/$\epsilon$/$\gamma$ in the higher layers. The generic propagation rule

$$\begin{aligned} R_k&= \sum _l \frac{a_k \cdot \rho (w_{kl})}{\epsilon + \sum _{0,k} a_k \cdot \rho (w_{kl})} R_l, \end{aligned}$$

of which LRP-0/$\epsilon$/$\gamma$ are special cases, can be rewritten as $R_k = a_k c_k$ with

$$ c_k(\varvec{a}) = \sum _l \rho (w_{kl}) \, \frac{\max \big (0,\sum _{0,k} a_k(\varvec{a}) \cdot w_{kl}\big )}{\epsilon + \sum _{0,k} a_k(\varvec{a}) \cdot \rho (w_{kl})} \, c_l(\varvec{a}), $$

where the dependences on lower activations $\varvec{a}$ have been made explicit. Assume $c_l(\varvec{a})$ to be approximately locally constant w.r.t. $\varvec{a}$. Because other terms that depend on $\varvec{a}$ are diluted by two nested sums, it is plausible that $c_k(\varvec{a})$ is again locally approximately constant, which is the assumption made by the relevance model $\widehat{R}_k(\varvec{a})$.