Safemotion Lib
Loading...
Searching...
No Matches
Public Member Functions | List of all members
fastreid.layers.activation.GELU Class Reference
Inheritance diagram for fastreid.layers.activation.GELU:

Public Member Functions

 forward (self, x)
 

Detailed Description

Paper Section 3.4, last paragraph notice that BERT used the GELU instead of RELU

Definition at line 53 of file activation.py.

Member Function Documentation

◆ forward()

fastreid.layers.activation.GELU.forward ( self,
x )

Definition at line 58 of file activation.py.

58 def forward(self, x):
59 return 0.5 * x * (1 + torch.tanh(math.sqrt(2 / math.pi) * (x + 0.044715 * torch.pow(x, 3))))

The documentation for this class was generated from the following file: