DNNs have also been proposed for direct discrimnative speaker classification, as witnessed by the recent literature on this topic [13–16]. Most of past attempts, however, employed hand-crafted features such as FBANK and MFCC coefficients [13, 17, 18]. These engineered features are originally designed from perceptual evidence and there are no guarantees that such representations are optimal for all speech-related tasks.
Standard features, for instance, smooth the speech spectrum, possibly hindering the extraction of crucial narrow-band speaker characteristics such as pitch and formants. To mitigate this drawback, some recent works have proposed directly feeding the network with spectrogram bins [19–21] or even with raw waveforms [22–34].
CNNs are the most popular architecture for processing raw speech samples, since weight sharing, local filters, and pooling help discover robust and invariant representations.
We believe that one of the most critical part of current waveform-based CNNs is the first convolutional layer. This layer not only deals with high-dimensional inputs, but is also more affected by vanishing gradient problems, especially when employing very deep architectures.
The filters learned by the CNN often take noisy and incongruous multi-band shapes, especially when few training samples are available. These filters certainly make some sense for the neural network, but do not appeal to human intuition, nor appear to lead to an efficient representation of the speech signal.
To help the CNNs discover more meaningful filters in the input layer, this paper
proposes to add some constraints on their shape. Compared to standard CNNs, where the filter- bank characteristics depend on several parameters (each element of the filter vector is directly learned), the SincNet convolves the waveform with a set of parametrized sinc functions that implement band-pass filters. The low and high cut- off frequencies are the only parameters of the filter learned from data. This solution still offers considerable flexibility, but forces the network to focus on high-level tunable parameters with broad impact on the shape and bandwidth of the resulting filter.
In standard CNNs, all the L elements (taps) of each filter are learned from data. Conversely, the proposed SincNet (depicted in Fig. 1) performs the convolution with a predefined function g that depends on few learnable parameters θ only.
The cut-off frequencies can be initialized randomly in the range[0, fs /2], where fs represents the sampling frequency of the input signal. As an alternative, filters can be initialized with the cutoff frequencies of the mel-scale filter-bank.
To ensure f1 ≥ 0 and f2 ≥ f1, the previous equation is actually fed by the following parameters:
Note that no bounds have been imposed to force f2 to be smaller than the Nyquist frequency, since we observed that this constraint is naturally fulfilled during training. Moreover, the gain of each filter is not learned at this level. This parameter is managed by the subsequent layers, which can easily attribute more or less importance to each filter output.
# Initialization
def __init__(self, N_filt,Filt_dim,fs):
super(sinc_conv,self).__init__()
# Mel Initialization of the filterbanks
low_freq_mel = 80
high_freq_mel = (2595 * np.log10(1 + (fs / 2) / 700)) # Convert Hz to Mel
mel_points = np.linspace(low_freq_mel, high_freq_mel, N_filt) # Equally spaced in Mel scale
f_cos = (700 * (10**(mel_points / 2595) - 1)) # Convert Mel to Hz
b1=np.roll(f_cos,1)
b2=np.roll(f_cos,-1)
b1[0]=30
b2[-1]=(fs/2)-100
self.freq_scale=fs*1.0
self.filt_b1 = nn.Parameter(torch.from_numpy(b1/self.freq_scale))
self.filt_band = nn.Parameter(torch.from_numpy((b2-b1)/self.freq_scale))
self.N_filt=N_filt
self.Filt_dim=Filt_dim
self.fs=fs
https://en.wikipedia.org/wiki/Mel_scale
A popular formula to convert f hertz into m mels is
The phase of the rect(·) function is considered to be linear.
for i in range(self.N_filt):
low_pass1 = 2*filt_beg_freq[i].float()*sinc(filt_beg_freq[i].float()*self.freq_scale,t_right)
low_pass2 = 2*filt_end_freq[i].float()*sinc(filt_end_freq[i].float()*self.freq_scale,t_right)
band_pass=(low_pass2-low_pass1)
band_pass=band_pass/torch.max(band_pass)
filters[i,:]=band_pass.cuda()*window
out=F.conv1d(x, filters.view(self.N_filt,1,self.Filt_dim))
return out
def flip(x, dim):
xsize = x.size()
dim = x.dim() + dim if dim < 0 else dim
x = x.contiguous()
x = x.view(-1, *xsize[dim:])
x = x.view(x.size(0), x.size(1), -1)[:, getattr(torch.arange(x.size(1)-1,
-1, -1), ('cpu','cuda')[x.is_cuda])().long(), :]
return x.view(xsize)
def sinc(band,t_right):
y_right= torch.sin(2*math.pi*band*t_right)/(2*math.pi*band*t_right)
y_left= flip(y_right,0)
y=torch.cat([y_left,Variable(torch.ones(1)).cuda(),y_right])
return y
y[n] = x[n] ∗ g[n, θ]
FIR滤波器的加窗是什么
An ideal bandpass filter (i.e., a filter where the passband is perfectly flat and the attenuation in the stopband is infinite) requires an infinite number of elements L. Any truncation of g thus inevitably leads to an approximation of the ideal filter, characterized by ripples in the passband and limited attenuation in the stopband. A popular solution to mitigate this issue is windowing [35]. Windowing is performed by multiplying the truncated function g with a window function w, which aims to smooth out the abrupt discontinuities at the ends of g:
.
This paper uses the popular Hamming window [36], defined as follows:
.
def forward(self, x):
filters=Variable(torch.zeros((self.N_filt,self.Filt_dim))).cuda()
N=self.Filt_dim
t_right=Variable(torch.linspace(1, (N-1)/2, steps=int((N-1)/2))/self.fs).cuda()
min_freq=50.0;
min_band=50.0;
filt_beg_freq=torch.abs(self.filt_b1)+min_freq/self.freq_scale
filt_end_freq=filt_beg_freq+(torch.abs(self.filt_band)+min_band/self.freq_scale)
n=torch.linspace(0, N, steps=N)
# Filter window (hamming)
window=0.54-0.46*torch.cos(2*math.pi*n/N);
window=Variable(window.float().cuda())
The Hamming window is particularly suitable to achieve high frequency selectivity [36]. However, results not reported here reveals no significant performance difference when adopting other functions, such as Hann, Blackman and Kaiser windows ( 几种常见窗函数的特性 ).
https://docs.scipy.org/doc/scipy-0.19.1/reference/generated/scipy.signal.hamming.html
def flip(x, dim):
xsize = x.size()
dim = x.dim() + dim if dim < 0 else dim
x = x.contiguous()
x = x.view(-1, *xsize[dim:])
x = x.view(x.size(0), x.size(1), -1)[:, getattr(torch.arange(x.size(1)-1,
-1, -1), ('cpu','cuda')[x.is_cuda])().long(), :]
return x.view(xsize)
def sinc(band,t_right):
y_right= torch.sin(2*math.pi*band*t_right)/(2*math.pi*band*t_right)
y_left= flip(y_right,0)
y=torch.cat([y_left,Variable(torch.ones(1)).cuda(),y_right])
return y
class sinc_conv(nn.Module):
def __init__(self, N_filt,Filt_dim,fs):
super(sinc_conv,self).__init__()
# Mel Initialization of the filterbanks
low_freq_mel = 80
high_freq_mel = (2595 * np.log10(1 + (fs / 2) / 700)) # Convert Hz to Mel
mel_points = np.linspace(low_freq_mel, high_freq_mel, N_filt) # Equally spaced in Mel scale
f_cos = (700 * (10**(mel_points / 2595) - 1)) # Convert Mel to Hz
b1=np.roll(f_cos,1)
b2=np.roll(f_cos,-1)
b1[0]=30
b2[-1]=(fs/2)-100
self.freq_scale=fs*1.0
self.filt_b1 = nn.Parameter(torch.from_numpy(b1/self.freq_scale))
self.filt_band = nn.Parameter(torch.from_numpy((b2-b1)/self.freq_scale))
self.N_filt=N_filt
self.Filt_dim=Filt_dim
self.fs=fs
def forward(self, x):
filters=Variable(torch.zeros((self.N_filt,self.Filt_dim))).cuda()
N=self.Filt_dim
t_right=Variable(torch.linspace(1, (N-1)/2, steps=int((N-1)/2))/self.fs).cuda()
min_freq=50.0;
min_band=50.0;
filt_beg_freq=torch.abs(self.filt_b1)+min_freq/self.freq_scale
filt_end_freq=filt_beg_freq+(torch.abs(self.filt_band)+min_band/self.freq_scale)
n=torch.linspace(0, N, steps=N)
# Filter window (hamming)
window=0.54-0.46*torch.cos(2*math.pi*n/N);
window=Variable(window.float().cuda())
for i in range(self.N_filt):
low_pass1 = 2*filt_beg_freq[i].float()*sinc(filt_beg_freq[i].float()*self.freq_scale,t_right)
low_pass2 = 2*filt_end_freq[i].float()*sinc(filt_end_freq[i].float()*self.freq_scale,t_right)
band_pass=(low_pass2-low_pass1)
band_pass=band_pass/torch.max(band_pass)
filters[i,:]=band_pass.cuda()*window
out=F.conv1d(x, filters.view(self.N_filt,1,self.Filt_dim))
return out
