This paper proposes a boundary operation technique of two-dimensional (2D) non-separable oversampled lapped transforms (NSOLT). The proposed technique is based on a lattice structure consisting of the 2D separable block discrete cosine transform and non-separable redundant support-extension processes. The atoms are allowed to be anisotropic with the oversampled, symmetric, real-valued, compact-supported, and overlapped property. First, the blockwise implementation is developed so that the atoms can be locally controlled. The local control of atoms is shown to maintain perfect reconstruction. This property leads an atom termination (AT) technique as a boundary operation. The technique overcomes the drawback of NSOLT that the popular symmetric extension method is invalid. Through some experimental results with iterative hard thresholding, the significance of AT is verified.

An infinite mixture model is applied to model-based speaker clustering with sampling-based optimization to make it possible to estimate the number of speakers. For this purpose, a framework of non-parametric Bayesian modeling is implemented with the Markov chain Monte Carlo and incorporated in the utterance-oriented speaker model. The proposed model is called the utterance-oriented Dirichlet process mixture model (UO-DPMM). The present paper demonstrates that UO-DPMM is successfully applied on large-scale data and outperforms the conventional hierarchical agglomerative clustering, especially for large amounts of utterances.

Vehicle license platerecognition in natural scene is an important research topic in computer vision. The license plate recognition approach in the specific scene has become a relatively mature technology. However, license plate recognition in the natural scene is still a challenge since the image parameters are highly affected by the complicated environment. For the purpose of improving the performance of license plate recognition in natural scene, we proposed a solution to recognize real-world Chinese license plate photographs using the DCNN-RNN model. With the implementation of DCNN, the license plate is located and the features of the license plate are extracted after the correction process. Finally, an RNN model is performed to decode the deep features to characters without character segmentation. Our state-of-the-art system results in the accuracy and recall of 92.32 and 91.89% on the car accident scene dataset collected in the natural scene, and 92.88 and 92.09% on Caltech Cars 1999 dataset.

The exponential growth in digital technology is leading us to a future in which all things and all people are connected all the time, something we refer to as The Infinite Network (TIN), which will cause profound changes in every industry. Here, we focus on the impact it will have in healthcare. TIN will change the essence of healthcare to a data-driven continuous approach as opposed to the event-driven discrete approach used today. At a micro or individual level, smart sensing will play a key role, in the form of embedded sensors, wearable sensors, and sensing from smart medical devices. At a macro or aggregate level, healthcare will be provided by Intelligent Telehealth Networks that evolve from the telehealth networks that are available today. Traditional telemedicine has delivered remote care to patients in the area where doctors are not readily available, but has not achieved at large scale. New advanced networks will deliver care at a much larger scale. The long-term future requires intelligent hybrid networks that combine artificial intelligence with human intelligence to provide continuity of care at higher quality and lower cost than is possible today.

Deep neural networks (DNN) have solved many tasks, including image classification, object detection, and semantic segmentation. However, when there are huge parameters and high level of computation associated with a DNN model, it becomes difficult to deploy on mobile devices. To address this difficulty, we propose an efficient compression method that can be split into three parts. First, we propose a cross-layer matrix to extract more features from the teacher's model. Second, we adopt Kullback Leibler (KL) Divergence in an offline environment to make the student model find a wider robust minimum. Finally, we propose the offline ensemble pre-trained teachers to teach a student model. To address dimension mismatch between teacher and student models, we adopt a $1\times 1$ convolution and two-stage knowledge distillation to release this constraint. We conducted experiments with VGG and ResNet models, using the CIFAR-100 dataset. With VGG-11 as the teacher's model and VGG-6 as the student's model, experimental results showed that the Top-1 accuracy increased by 3.57% with a $2.08\times$ compression rate and 3.5x computation rate. With ResNet-32 as the teacher's model and ResNet-8 as the student's model, experimental results showed that Top-1 accuracy increased by 4.38% with a $6.11\times$ compression rate and $5.27\times$ computation rate. In addition, we conducted experiments using the ImageNet$64\times 64$ dataset. With MobileNet-16 as the teacher's model and MobileNet-9 as the student's model, experimental results showed that the Top-1 accuracy increased by 3.98% with a $1.59\times$ compression rate and $2.05\times$ computation rate.

In some security applications, it is important to transmit just enough information to take the right decisions. Traditional video codecs try to maximize the global quality, irrespective of the video content pertinence for certain tasks. To better maintain the semantics of the scene, some approaches allocate more bitrate to the salient information. In this paper, a semantic video compression scheme based on seam carving is proposed. The idea is to suppress non-salient parts of the video by seam carving. The reduced sequence is encoded with H.264/AVC while the seams are encoded with our approach. The main contributions of this paper are (1) an algorithm that segments the sequence into group of pictures, depending on the content, (2) a spatio-temporal seam clustering method, (3) an isolated seam discarding technique, improving the seam encoding, (4) a new seam modeling, avoiding geometric distortion and resulting in a better control of the seam shapes, and (5) a new encoder which reduces the overall bit-rate. A full reference object-oriented quality metric is used to assess the performance of the approach. Our approach outperforms traditional H.264/AVC intra encoding with a Bjontegaard's rate improvement between 7.02 and 21.77% while maintaining the quality of the salient objects.

An algorithm for adaptive Mel frequency cepstral coefficients (MFCC) feature truncation is proposed to improve automatic speech recognition (ASR) performance under acoustically mismatched conditions. Using the relationship found between MFCC base feature truncation and degree of acoustic mismatch of speech signals with respect to recognition models, the proposed algorithm performs utterance-specific MFCC feature truncation for test signals to address their acoustic mismatch in context of ASR. The proposed technique, without any prior knowledge about the speaker of the test utterance, gives 38% (on a connected-digit recognition task) and 36% (on a continuous speech recognition task) relative improvement over baseline in ASR performance for children's speech on models trained on adult speech, which is also found to be additive to improvements obtained with vocal tract length normalization and/or constrained maximum likelihood linear regression. The generality and effectiveness of the algorithm is also validated for automatic recognition of children's and adults' speech under matched and mismatched conditions.

A frame quality control mechanism for H.264/AVC is proposed in this research. The research objective is to ensure that a suitable quantization parameter (QP) can be assigned to each frame so that the target quality of each frame will be achieved. One of the potential application is consistently maintaining  frame quality during the encoding process to facilitate video archiving and/or video surveillance. A single-parameter distortion to quantization (D–Q) model is derived by training a large number of frame blocks. The model parameter can be determined from the frame content before the exact encoding process. Given the target quality for a video frame, we can then select an appropriate QP according to the proposed D–Q model. Model refinement and QP adjustment of subsequent frames can be applied by examining the coding results of previous data. Such quality measurements as peak signal to noise ratio (PSNR) and structural similarity (SSIM) can be employed. The experimental results verify the feasibility of the proposed constant quality video coding framework.

With wider luminance range than conventional low dynamic range (LDR) images, high dynamic range (HDR) images are more consistent with human visual system (HVS). Recently, JPEG committee releases a new HDR image compression standard JPEG XT. It decomposes an input HDR image into base layer and extension layer. The base layer code stream provides JPEG (ISO/IEC 10918) backward compatibility, while the extension layer code stream helps to reconstruct the original HDR image. However, this method does not make full use of HVS, causing waste of bits on imperceptible regions to human eyes. In this paper, a visual saliency-based HDR image compression scheme is proposed. The saliency map of tone mapped HDR image is first extracted, then it is used to guide the encoding of extension layer. The compression quality is adaptive to the saliency of the coding region of the image. Extensive experimental results show that our method outperforms JPEG XT profile A, B, C and other state-of-the-art methods. Moreover, our proposed method offers the JPEG compatibility at the same time.

This paper presents the origin of digital information devices, the Silicon Audio, and its family. The Silicon Audio is the digital counterpart of the Walkman and the ancestor of the iPod. It employs the MPEG / Audio Layer II algorithm for data compression, which was standardized by ISO (International Standardization Organization)/ IEC (International Electrotechnical Commission). A semiconductor memory card is equipped with to store the compressed signal. Since it has no mechanical movement, it is robust against shocks and vibrations that had been a serious problem for portable audio players. The background of the development, implementations, challenges toward a commercial product, and impact on audio players as well as personal information devices are discussed with its family including a video derivative, the Silicon View.

Over 20 years have passed since a free-viewpoint video technology has been proposed with which a user's viewpoint can be freely set up in a reconstructed three-dimensional space of a target scene photographed by multi-view cameras. This technology allows us to capture and reproduce the real world as recorded. Once we capture the world in a digital form, we can modify it as augmented reality (i.e., placing virtual objects in the digitized real world). Unlike this concept, the augmented world allows us to see through real objects by synthesizing the backgrounds that cannot be observed in our raw perspective directly. The key idea is to generate the background image using multi-view cameras, observing the backgrounds at different positions and seamlessly overlaying the recovered image in our digitized perspective. In this paper, we review such desired view-generation techniques from the perspective of free-view point image generation and discuss challenges and open problems through a case study of our implementations.

This paper focuses on applications of Bayesian approaches to acoustic modeling for speech recognition and related speech-processing applications. Bayesian approaches have been widely studied in the fields of statistics and machine learning, and one of their advantages is that their generalization capability is better than that of conventional approaches (e.g., maximum likelihood). On the other hand, since inference in Bayesian approaches involves integrals and expectations that are mathematically intractable in most cases and require heavy numerical computations, it is generally difficult to apply them to practical speech recognition problems. However, there have been many such attempts, and this paper aims to summarize these attempts to encourage further progress on Bayesian approaches in the speech-processing field. This paper describes various applications of Bayesian approaches to speech processing in terms of the four typical ways of approximating Bayesian inferences, i.e., maximum a posteriori approximation, model complexity control using a Bayesian information criterion based on asymptotic approximation, variational approximation, and Markov chain Monte Carlo-based sampling techniques.

Spectrum scarcity has driven enhancements of Long-Term Evolution (LTE) in utilizing unlicensed bands in conjunction with licensed bands for delivering mobile data, resulting in the introduction of LTE unlicensed technologies such as Rel-13 LTE–Licensed-Assisted Access (LAA), Rel-14 LTE–Enhanced Licensed-Assisted Access (eLAA), and LTE-Unlicensed (LTE-U). The next-generation radio access technology, 5G New Radio(NR), faces greater technical challenge due to the need to support frequency bands covering various spectrum licensing regimes and a wide range of frequencies (up to 100 GHz) with very different signal propagation characteristics. This paper presents an overview of LAA and eLAA technical features and 5G NR design considerations to achieve a unified access in licensed and unlicensed bands.

Voice conversion aims to change a source speaker's voice to make it sound like the one of a target speaker while preserving linguistic information. Despite the rapid advance of voice conversion algorithms in the last decade, most of them are still too complicated to be accessible to the public. With the popularity of mobile devices especially smart phones, mobile voice conversion applications are highly desirable such that everyone can enjoy the pleasure of high-quality voice mimicry and people with speech disorders can also potentially benefit from it. Due to the limited computing resources on mobile phones, the major concern is the time efficiency of such a mobile application to guarantee positive user experience. In this paper, we detail the development of a mobile voice conversion system based on the Gaussian mixture model (GMM) and the weighted frequency warping methods. We attempt to boost the computational efficiency by making the best of hardware characteristics of today's mobile phones, such as parallel computing on multiple cores and the advanced vectorization support. Experimental evaluation results indicate that our system can achieve acceptable voice conversion performance while the conversion time for a five-second sentence only takes slightly more than one second on iPhone 7.

The learning tools necessary to prepare the next generation of students must be shaped by the socio-economic needs of the 21st century. The needs of the 21st century – from rebuilding city scale physical infrastructure to personalized healthcare – not only require learning from the wealth of global information available on the Internet, but also the building of a strong grounding in fundamentals. History has shown that the depth in fundamentals has been achieved through conventional books. Indeed, authoritative books in physical fundamentals have been penned in the 19th century and early 20th century. We present a 21st century cyber-physical learning platform that combines the best of physical books with information systems. The systemic instantiation of the platform combines modern optics and computing to view books, scan objects, and enable interactive learning – while simultaneously benefitting from the vast pool of information on the Internet. The hybrid learning platform preserves the best of the past to pave the way for the future. It also enables future research such as meta-data and descriptive tagging of the large number of images available on the web.

Visual retrieval and classification are of growing importance for a number of applications, including surveillance, automotive, as well as web and mobile search. To facilitate these processes, features are often computed from images to extract discriminative aspects of the scene, such as structure, texture or color information. Ideally, these features would be robust to changes in perspective, illumination, and other transformations. This paper examines two approaches that employ dimensionality reduction for fast and accurate matching of visual features while also being bandwidth-efficient, scalable, and parallelizable. We focus on two classes of techniques to illustrate the benefits of dimensionality reduction in the context of various industrial applications. The first method is referred to as quantized embeddings, which generates a distance-preserving feature vector with low rate. The second method is a low-rank matrix factorization applied to a sequence of visual features, which exploits the temporal redundancy among feature vectors associated with each frame in a video. Both methods discussed in this paper are also universal in that they do not require prior assumptions about the statistical properties of the signals in the database or the query. Furthermore, they enable the system designer to navigate a rate versus performance trade-off similar to the rate-distortion trade-off in conventional compression.

Social rhythms have been considered as relevant to mood disorders, but detailed analysis of social rhythms has been limited. Hence, we aim to assess social rhythms via social media use and predict users' psychiatric symptoms through their social rhythms. A two-wave survey was conducted in the Pigg Party, a popular Japanese avatar application. First and second waves of data were collected from 3504 and 658 Pigg Party users, respectively. The time stamps of their communication were sampled. Furthermore, the participants answered the General Health Questionnaire and perceived emotional support in the Pigg Party. The results indicated that social rhythms of users with many social supports were stable in a 24-h cycle. However, the rhythms of users with few social supports were disrupted. To predict psychiatric symptoms via social rhythms in the second-wave data, the first-wave data were used for training. We determined that fast Chirplet transformation was the optimal transformation for social rhythms, and the best accuracy scores on psychiatric symptoms and perceived emotional support in the second-wave data corresponded to 0.9231 and 0.7462, respectively. Hence, measurement of social rhythms via social media use enabled detailed understanding of emotional disturbance from the perspective of time-varying frequencies.

This paper presents an evaluation of parallel voice conversion (VC) with neural network (NN)-based statistical models for spectral mapping and waveform generation. The NN-based architectures for spectral mapping include deep NN (DNN), deep mixture density network (DMDN), and recurrent NN (RNN) models. WaveNet (WN) vocoder is employed as a high-quality NN-based waveform generation. In VC, though, owing to the oversmoothed characteristics of estimated speech parameters, quality degradation still occurs. To address this problem, we utilize post-conversion for the converted features based on direct waveform modifferential and global variance postfilter. To preserve the consistency with the post-conversion, we further propose a spectrum differential loss for the spectral modeling. The experimental results demonstrate that: (1) the RNN-based spectral modeling achieves higher accuracy with a faster convergence rate and better generalization compared to the DNN-/DMDN-based models; (2) the RNN-based spectral modeling is also capable of producing less oversmoothed spectral trajectory; (3) the use of proposed spectrum differential loss improves the performance in the same-gender conversions; and (4) the proposed post-conversion on converted features for the WN vocoder in VC yields the best performance in both naturalness and speaker similarity compared to the conventional use of WN vocoder.

The development of colorization algorithms through deep learning has become the current research trend. These algorithms colorize grayscale images automatically and quickly, but the colors produced are usually subdued and have low saturation. This research addresses this issue of existing algorithms by presenting a two-stage convolutional neural network (CNN) structure with the first and second stages being a chroma map generation network and a refinement network, respectively. To begin, we convert the color space of an image from RGB to HSV to predict its low-resolution chroma components and therefore reduce the computational complexity. Following that, the first-stage output is zoomed in and its detail is enhanced with a pyramidal CNN, resulting in a colorized image. Experiments show that, while using fewer parameters, our methodology produces results with more realistic color and higher saturation than existing methods.

In this paper, we present a technique that is known as Residual Sign Prediction in Transform Domain (TDRSP) and is aimed at increasing compression performance by reducing the bits overhead of residue sign. These signs are typically coded by entropy coders in bypass mode that results in the high cost of sign bins which require 1 bit per bin in a bitstream. TDRSP allows us to reduce this cost by predicting residue signs so that the probability of a guess is significantly higher than 50%. Hence, arithmetic coding with contexts becomes applicable to the signs. In contrast to Residual Sign Prediction (RSP) performed in spatial domain, TDRSP avoids switching between domains and carries out calculations completely in transform domain to efficiently decrease the computational complexity of RSP. Simulations performed on top of Versatile Video Coding test model reference software (VTM-1.0) in accordance with the Joint Video Experts Team common test conditions show that more than 2.0% and up to 1.8% of the Bjøntegaard Delta rate can be achieved for All Intra and Random Access configurations, respectively.