121) -> float:
122 """
123 Return the learning rate warmup factor at a specific iteration.
124 See https://arxiv.org/abs/1706.02677 for more details.
125 Args:
126 method (str): warmup method; either "constant" or "linear".
127 iter (int): iteration at which to calculate the warmup factor.
128 warmup_iters (int): the number of warmup iterations.
129 warmup_factor (float): the base warmup factor (the meaning changes according
130 to the method used).
131 Returns:
132 float: the effective warmup factor at the given iteration.
133 """
134 if iter >= warmup_iters:
135 return 1.0
136
137 if method == "constant":
138 return warmup_factor
139 elif method == "linear":
140 alpha = iter / warmup_iters
141 return warmup_factor * (1 - alpha) + alpha
142 else:
143 raise ValueError("Unknown warmup method: {}".format(method))
