| |
|
|||||||||||||||||||||||||
![]() |
||||||||||||||||||||||||||
05-06-01I wrapped up my experiment, found the signal to noise ratios and wrote my semester summary paper. All the info can be found in there.
04-30-01I ran my naive beam former to find the lowest signal to noise ratio. The lowest value I got was about 5157. This number is really just the average difference between the control microphone and the average of the other seven mics for each sample. The SNR seems a little high, but the magnitude of each mic went up to 20,000, so the SNR is a fourth of that, which may not be THAT bad. Here is the graph of the seven mics' averages when they are delayed to the lowest SNR.
The output of the mic we are delaying from:
Synthetic mixes (mics 1-3)
04-23-01ProgressThis week has been very productive for not having the microphone array available to use. First, to address last week's problem with file manipulation, that problem was sorted out and now the input and output of my C functions seem to be working smoothly. I moved on to begin developing a few little tests and experiments that will hopefully help anyone who may do future research with the array. Matlab: I wrote code which takes an audio file as input, runs a filter over the unsigned char format and converts it into a matlab-readable integer array. I also wrote a script which opens matlab, loads and graphs the array, and saves the array as a .jpg picture for later viewing. I tested this on the data that is produced by my sine wave production function and the graph looked like the appropriate sine wave. It was not completely constant, but I think it may be due to mis-sampling....if the 44.1KHz was increased, the graph would look like a perfect sine wave. I am confident that this code works. Synthetic Waves: I also wrote code which simulates two independent sound sources in a room. The code basically stalls each wave by the appropriate amount depending on the locations I set for each microphone in the array. I finished this code yesterday and I am not sure if it is correct yet. But I should be able to complete it tomorrow. Problems: I came across some compilation issues with the various drivers and included files, but I think I sorted out what was needed and thankfully was able to transport my code and use it on the graduate lab computers. There are several files from the OSS library that are not always standard. Also, I am not sure if my sin wave generation is correct. The output through a sound card sounds like a pure sin wave tone, but when I graph my reading on Matlab...it is short of a sine wave. Below is its graph:
It is close to a wave, perhaps sampling at 44.1KHz cannot capture the entire wave, maybe it is undersampling? GoalsWith the matlab conversion code completed and my simulation code basically done, I will be able to graph the simulation and also separate the waves as best I can. My main goal this week will be to run the beam-forming algorithm (which I completed this week as well) on the simulation data I have created. Hopefully I can draw out the two simulated sine waves. Also, Brandon and I had discussed analyzing the background noise and seeing if we can predict what it will look like in the future. With the matlab conversion, hopefully I can get to a microphone and take a look at it. I can see where the fallout point is on distance vs. noise. Since this is basically the last week I have to code, I hope to get some actual data! After this week, I will need to spend time documenting my code so future people will be able to use it affectively.
04-16-01ProgressProgressThis week was spent coding out an efficient way to set up the microphone array for an initial experiment. I am thinking of using one control microphone which will be positioned directly in front of my output speakers while the microphone array will be some distance from the speakers. At first I was just going to use the headset microphone which is attached to the generic sound card, but this would not be very accurate since the microphone array is set up with 8 high quality microphones. In order to maintain consistency, I think we need the same type of microphone as the control and as the components of the array. However, this leaves us one microphone short for the array. So as of now, the setup has one microphone (mic 0) next to the speaker and mics 1-7 are on the array. The code I wrote simply reads an audio file from the disk. This makes it easy to run the experiment because I can just pipe the audio input directly to a file....the actual analysis can be done off-line later. I have written functions which take distances between mics and calculate the number of samples that must be delayed, based on the current frequency and the speed of sound. Also, the main function which parses the audio file and computes the signal to noise ratio is completed. However, by 'completed' I mean it all looks ok to me as the programmer. Since the computer is now at the Center for Future Health, I cannot test the code. I tried making my own 'audio' and storing it to a file, only to read it later by my code....thereby testing it. But I am running into extreme difficulty with files in C and unsigned char types. My code runs through a loop and is supposed to be printing sequential hexadecimal numbers to file...but when I read the stuff back in, it is all garbled. I am still trying to figure out what is wrong. So to summarize, my code for the experiment is completed, but it is not yet tested and ready to go. GoalsObviously I hope to solve my file input/output problems and to verify that my C code is correct. Once correct, we are ready to run some experiments and obtain results. But once at that stage, it all depends on when the new computer arrives for the microphone arrays. Also, I can experiment with different array setups to get the maximum results. Hanging the microphones from the ceiling is one possibility, but the sound may be too faint from such positions.
04-09-01ProgressLast week I discovered that there are still pops and/or breaks in the audio even after I implemented streams in C. I have more details on why this may be occuring. First, there is a second sound card in the computer, a more 'generic' one which should get along with the rest of the hardware. When I use the same code on this card as opposed to the STUDI/O, input from the microphones is crystal clear without pops. It is only with the STUD that we get breaks in the input. Therefore, it is pretty safe to conclude that the problem is not with my code but with some hardware issue. I ran my program without writing the inputted audio to the hard drive also. The streams ran as usual, but I just skipped the disk write step. It turns out that the input was pop-free in this case. I e-mailed OSS support and they responded with a couple suggestions, one of which was to change the input sync from internal to external, but that did not help. The other suggestion was that the disk and card are conflicting and to try switching the drive to DMA mode. Unfortunately, our drive does not support DMA and we cannot try it. But all evidence points to a disk-card conflict that occurs upon writing. I am going to ignore the breaks for now and hope that a new computer will eliminate the problem. Brandon wants me to try reading only into memory and skip the disk for now. I have run /proc/meminfo a couple times and there seems to be 8 MB free in general. That should be enough.
Mid-way through the above fiasco this week, I also began coding some functions that will be able to input and analyze the signals. I am working on getting the signal-to-noise ratio of one microphone compared to all eight averaged together. The code is almost finished and will need to be tested to make sure the numbers are correct. I am still working on making it easy to run these tests. Unfortunately it is difficult to create all-purpose functions because experiments can be so varied. I still need to see how outputting from the Crystal Sound Card and inputting from the STUDI/O is going to work... GoalsThis week will be mostly coding what I just discussed and hoping that everything works out the way I want. Inputting and outputting simultaneously might pose a problem. If all goes well, I think I can obtain some numerical results from the array either next week or at the end of this week.
04-02-01ProgressI completed all conversion from C++ to C of my working code. The main task was to create the appropriate functions that C++ was able to use as class member functions and to keep everything organized in a manner that will not become too confusing for me (or anyone else looking at the code). I believe everything is organized well and a short writeup at the end of the semester would clarify anything else that might seem confusing to others. The threads for inputting from the microphones are finally working in C also. Input from the mics and writing to the hard disk alternate between two buffers, keeping the gaps in the audio to a minimum. However, it is impossible to tell if the threads are really working. There are still consistent pops in the audio after recording. At first I thought that it was one of two problems: first, maybe the threads really are not working, and second, it might just be that my output function (not threaded) is creating them. However, I don't think it is either. During recording of my voice, the two speakers attached to the STUDI/O output stream the audio back out while I talk. It is a direct line from the microphones to the speakers. During recording, you hear your voice. It is also during recording that these same pops are appearing. So the output from the microphones has the same pattern as the input. I am not sure what to conclude from this. When I am not running my code on the microphone input, the output from the speakers is pretty clear of any sort of static and pops. The pops appear the instant I hit record. For this reason, I think the process of reading from the STUDI/O card is still creating a lag on the input. Even with threads, the input is still distorted. Well it is not distorted, it just has short breaks in the audio. While this is bad, I do not think it is a hindrance from pursuing experimentation. If all the eight microphones have the same breaks in the input, the error cancels out. ---- I also wrote some code that runs a sine wave generator. The function sends a sine wave in any frequency to the audio output device. This will be handy in preliminary audio tests and experiments. GoalsI would like to find out why we cannot get clear input and output from the microphones, it does not seem to be such a hard task, but yet it just doesn't seem to work. Also, I plan to set up a first experiment with some microphones and my sine wave generator. It may be difficult to output the wave and input simultaneously, so I may have to do a bit more coding to accomplish that. But I hope to get some comparisons between using one microphone as opposed to several.
03-26-01ProgressI spent the entire week attempting to convert all of my C++ code into C. The reason that began this was because I cannot use pthreads very well within C++ apparently. And upon Brandon's recommendation that C is more veratile for people using this type of stuff, I tried to convert to C. At first I thought it should not be too difficult of a task because I do not have a significant amount of code, although there is quite a bit. Unfortunately, this task proved to be very very difficult. I have my original C++ very object-oriented and did not realize how much would have to change. But I made the changes and have de-modularized everything appropriately. And I whittled down all of the initial errors until my code finally compiled under C successfully. However, I still have not been able to get the code to run without segmentation faulting. I have verified that variables are being set correctly and whatnot, but the program seems to seg-fault on exit from the main function. I've been debugging this for a couple of days now and still have not found the source of this problem. gdb debugger shows the problem to be on exit from main. I decided to go back to my old C++ code and see if I could at least get the threads to work on it...programming this in C was a pain and I want to see if the threads will actually work, otherwise this would all be a waste. That is all there is to report really, I've spent many hours converting this stuff over and trying to debug it. GoalsSame as last week I guess. I want this C to finally work so we can get on to some actual progress and work. It seems so trivial and we should be able to start inputting data and running experiments.
03-19-01ProgressI worked on the input format that the audio device returns at the beginning of this week. It's just a little nuisance to convert the unsigned char format into integers, but you have to be careful in order to do it correctly. I think it is correct right now, but I have not done any actual operations on the numbers yet. Also, I wrote some code that is able to extract an individual microphone out of a file of multiple channel inputs. I have recorded with three microphones and was able to playback each individually. This leads to the obvious extension of comparing the numerical values and creating noise-to-signal ratios which we want to do eventually. We are ready to begin actual recording, but I have been noticing little hiccups in the recordings that I passed off as static before. After listening to longer recordings, the breaks are too regular. I altered the buffer size that I am using to input the data and found that larger buffers lead to less hiccups but longer hiccups. And of course, smaller buffers result in many short hiccups. I was hoping the process of writing to disk would be quick enough that the break was inconsequential, but it doesn't seem to be good enough. I have spent the second half of this week learning about threads in C and how to coordinate in order to read and write through two buffers instead of one. I have developed a base understanding of the create, join, mutex, etc. functions and I wrote some code that should work to record all the audio we need without breaks. However, the code is not working and something is going on with the threads which is invisible to me. I create functions that are completely empty and I get segmentation faults once I pass them to a thread. It may have something to do with the fact that they are class functions, but I do not see why. Also...I documented the hardware we are using and how we set it up in a short readme. In addition, I documented the steps I went through to get the STUDI/O working properly in a second readme. Both are located in /micarray/doc. GoalsShort term, I hope to debug the code and see what is wrong with the threads. This could actually take a while since I cannot find documentation on this kind of error. Medium term: If I get the threads working, I believe I will have all the requirememnts to actually making use of this hardware. I can begin testing if 8 microphones are better than 1. I might compress my code and turn some of it into a library first, making everything easier to work with. The code is beginning to get lengthy. Long term: As you can see, long term goals have become medium term. So that's good. Research areas such as focusing in on a given point in 3D space might be possible. Or the opposite which is finding a source point in 3D from the input.
03-12-01ProgressI was able to get in contact with OSS support and they suggested adding a line to the options.cfg file: "The problem is that your card is not in ADAT mode that is required. You need to add a line containing studio_mode=1 to options.cfg. Please take a look at /usr/lib/oss/docs/README.Studio." So this was supposed to be 'obvious' apparently...due to the curtness of her reply. But the readme is far from that. However, the change seems to have worked and the driver is functioning very well. There is direct input and output from the microphone sound board to the card and back out to the speakers. Also, the change has opened up more input and outputs that are recognized by /dev/sndstat. There are 10 outputs and 10 inputs as well as a single monitor ouptut (stereo). My code worked for input off of channel 13 and output through channel 3 straight to the speakers. It works great. There is one problem which I have not been able to solve yet. I cannot output through channel 2, the STUD monitor output port. The system prints out an error message concerning an active port and the inability to use it. It might be because I am currently only inputting one channel and that port requires two, but I have not yet tested double or more microphone inputs at once. Goals:I believe the STUDI/O card is now functioning appropriately for the tasks we wish to complete. So I plan on beginning preparations for actual experiments and such. The first goal is to finally figure out how the input is stored on the computer. It is buffered with unsigned characters, but I do not know how to convert these to integers and/or compare them to each other in order to develop noise ratios and the like. So I am going to research those areas on the net and hopefully find the answer quickly. Also, I will look into hooking up more than one microphone and writing code that can parse audio files easily. One task will be comparing a single microphone out of a file containing 8 microphone readings. Parsing shouldn't be too difficult, just a little tedious to code. Long term has not changed and I think we can accomplish some good experiments. Noise ratios between a single mic and the full array are not far off.
02-19-01Progress:I worked on two tasks this (short) week. The first of which was to alter my C++ code and allow for separate input and output devices. Also, it wasn't very modularized last week, so I corrected that and the options available are simple to use. Recording and playback are easy functions to call with all error checks covered as of now. The software works on the lab computer's basic soundcard. I am able to get input and output through the headset microphone and speakers. The second task I attempted was to finally interface with the STUDI/O card and get input from the eight microphones. This is proving more difficult than it sounds. The documentation that OSS provides for reading audio in is very sparse. There is a directory full of readme's, but no real examples or explanations of how to actually read input is given. I did verify that the card is recognized. By 'cat'ing /dev/sndstat, you can get a list of the devices that are available for use. There are two for the original soundcard, four inputs and four outputs for the STUDI/O, and eight other devices for some other purpose that I can't figure out. We hooked up the speakers to the output of the STUDI/O and had trouble getting that to even work. After a while though, I finally wrote to the proper device and got output from the STUD. So I know the card is working and connected properly. Devices /dev/dsp3-6 are available for output. But the output is kind of strange. I can only get output from the right speaker and the left speaker is always silent. Each dsp from 3 to 6 outputs to the right speaker...I can't explain this. Unfortunately, input is still a mystery. You cannot output to the dsp7-10 which are supposed to be the inputs...so that is good...the system returns an error when you try to open the device file for write only priveleges. But I still cannot read anything from it. Goals:Obviously my main goal is to read from the microphone array...I don't have any clear direction on this except to keep messing around with it. The other thing I wanted to look into was synchronization of input. As it stands, I read from the microphone and then write to a file...once writing to a file is completed, I resume reading from the mic. This would leave a short gap in the input which is undesirable. So I need to get the microphone to continuously read.
02-12-01ProgressThis week I was able to solve the bad input my microphone was giving me last week. Linux sound API allows you to change sampling rate, sample format, and the number of channels to read from. Earlier, I was inputting using the linux default for my sound card. This turned out to be a sample format of 8 bits and a sampling rate of 22KHz. I changed the rate up to 44.1KHz, but the input was still extremely fuzzy and loud. My system supported a 16 bit sample format (I also found out how to test what your system supports), so I tried 16 bits and the input was perfect. I was able to input one second of speech into a temporary buffer of unsigned char's, and then write it back to the audio device. You can write a simple imitation program that will do this endlessly, repeating what you say. By the time I got some good input, I had figured out the basics of the linux sound interface and I have a couple good functions that can test what the system is capable of, and read in good audio. However, audio quickly fills up buffer space and my PC could not handle more than one and a half seconds of 16 bit audio at 44KHz with only one channel (not stereo). So my next goal is to try and write the buffer to a file once it is full, and continue reading from the audio device. I attempted it briefly this week and was able to record multiple seconds of input...but I'm not sure if it is accurate. There may be a gap between the times that the buffer fills and I append it to a file, till the next time I read into the buffer. Also this week, I began on the class hierarchy that will be needed for all this code. I created an abstract base audio class from which I derived a LinuxAudio class which will interface with linux. Hopefully it will be robust enough to be cross platform, and the only part another programmer needs to code is a WindowsAudio class (for example) and everything else will be functional. My linuxaudio class is currently working for brief input. GoalsNext Week: Since the microphone input is progressing along so quickly, I hope to figure out how to store microphone input to file without leaving gaps from the latency of writing to disk. Once this is accomplished, I should be ready for any length of input (within reason). Also, I want to try this on the lab machine now...and get the interface with the soundboard running. Longer Term: I still need to figure out how to read the input from the microphone. It is in unsigned char format in the buffer...I'm not clear on how to read that mathematically. Long Term: The goals are still the same. Input from eight microphones looks very promising as it stands now. And analysis from them should be possible
02-05-01ProgressI worked solely on my personal PC this week in order to establish a basis for interacting with a soundcard. The priveleges needed to install sound modules and such made it easier to simply stay at home. Right now I'm just trying to learn how to interface, so there is no real need to use the lab computer yet. I have been successful in installing the appropriate driver for my sound card and I have learned how to install and uninstall modules on top of the linux kernel. Once I was able to obtain some audio (mp3 player and other applications proved it was working), I began to investigate the linux sound API. It turns out that there are actually two major groups working on developing a solid API for audio. The first and probably more well known is OSS. They have the most drivers for soundcards and have shipped with every version of linux to date. However, a group called ALSA (Advanced Linux Sound Architecture) also has their own API and apparently are gaining strength. The next red hat might ship with ALSA instead of OSS. The issue of future compatibility obviously comes up here. I don't know the workings of either OSS or ALSA, but I need to pick which libraries I will be using. I decided to go with OSS for a couple of reasons. One, we already have (or supposedly have) the driver for our STUDI/O card...made by OSS. Searching for another from ALSA could become a headache. Also, OSS is present on red hat already and most other unix machines, so it seems the right choice for now. And even if ALSA takes over in a few years, I doubt OSS will be dropped...compatibility will still be there hopefully. After deciding to go with OSS, I found a couple places on the Net to research how to interface with different cards. And so far I have been very successful (for a week's time at least). I have written a short C++ program which reads in from a microphone and then plays the reading directly after the recording. It took a while to get it going (the details of the API probably aren't suitable for this log :) ), and the recording is very poor. Actually, it's just loud noise...something is wrong. However, I can detect changes in pitch, so something is definitely being recorded. My sampling rate or a number of other things might be incorrect which is giving horrible output. FutureSo I got some input and output briefly. Short term for this week, I want to find out what is going on and get a better handle on this sound API stuff. Yeah, I just want some clear input and output. Medium term I hope to write a framework for the overall microphone array. In other words, the class structure and functions that I will need to read eight microphones. The challenge will be to do it without being system-dependent...and of course actually getting all eight mics to give me input. Long term I still think we can get all eight mics in sync. The sound API isn't completely clear, but I think I can figure out how to do it. Then we'll get some real analyses.
01-31-00This has been one of those slow weeks where lots of time is spent, but nothing seems to be learned. I ran through the OSS writeup of their STUDI/O driver. The driver creates buffer files from each input line and you need to write code that reads from the buffers as soon as they are filled...this seems to be the standard for reading in audio input. However, it is unclear whether the driver will support reading multiple mics at the same time. The documentation indicates vaguely that this option is not available until a future release of the driver. However, the documentation also discusses memory issues with reading simultaneous microphones. So that infers that it is possible to do such input... I e-mailed support at OSS and have not heard back yet. I found some more audio pages on Linux tools. One page has quite a few tools, and there is even a link for one tool that may be able to read 8 mics in sync. Unfortunately the link was broken. But some of the tools might prove useful and avoid hand-coding what is needed to do analysis on the microphone array. I got my hands on some code in C++ off the net which manipulates wave files and extracts needed information from them as well as writes to wave files. The functions and classes may be able to be used once we get microphone input. The wave format is the most popular and it might be useful to save to .wav and play it back on conventional audio programs. Next plan...I hope to read up on drivers and interfacing with hardware. My linux knowledge is pretty basic, I know basic commands and how to navigate, but when it comes to setting up hardware and linux organization, I need more knowledge. So I will read up on that, hopefully I can get a login on the lab machine, but I would need administer privileges. Maybe I can use my linux in my room to try things out, but then I need other drivers.... Anyway, I'm going to try to learn that stuff next. And if that goes well, then I will try out some of these linux tools and see if I can't get them to read from the single input sound card in the lab machine.
01-24-00I figured out how to record under the Cakewalk Demo program. There were just basic settings that needed to be configured together in order to get any input. Also, I discovered that the microphone inputs are read as 1L and 1R, 2L and 2R, etc. On the mixer board, channels 1 and 2 go to Mic 1Left and Mic 1Right respectively. So the mixer board has 16 channels, for 8 inputs to the STUD audio card. This is the reason I was only hearing sound out of the left speaker or left headphone, the input was not stereo. So in order to read in two separate microphones, use slots 1 and 3 on the mixer board. The Cakewalk program is excellent in that it records multiple microphones simultaneously and stores the wave pattern in sync with each other. I tried it on two mics, but eight would be simple as well. Unfortunately, this demo program does not allow saving of your data. So it is pretty much useless. I began looking into Windows API on reading in audio from standard audio inputs. There seems to be the capability in various wave* functions and structs that are listed in the help menus of Visual C++. Input works by filling up a buffer and setting a flag when the buffer is full. Then a program would access this buffer and save/manipulate the data until the buffer is filled again, etc. But the structure of how all this is accessed is vague and seems to be above me. I'm trying to understand some pages on the web I've found that I listed here. Hopefully I can translate it eventually... Finding programs for Windows to read multiple audio inputs in sync are hard to come by.
01-17-00I began working with the 8 microphone array under a WindowsNT interface. The sound card and accompanying software is working and I can record through normal Windows software. The STUD I/O came with several demo versions of audio software which I hoped would allow me to record all 8 microphones simultaneously. Unfortunately, all of them except one only record one microphone at a time or are for MIDI composing only. The 'Goldwave' program is the best for editing single recordings, but only 'Cakewalk' has the ability to record from multiple inputs in parallel. However I have not figured out how to use Cakewalk appropriately. The capability seems to be there, but it is not reading any of the input from the STUD card. The drivers are recognized, and the STUD ports show up in their drop down menus, but nothing is recorded. Help files are useless with the program, I hope to find some online or find another program that will allow recording in sync. Also, assuming I can find something, I also need to find a program that translates wave files into numeric recordings for analyzation. So far, work has been and will be a search mission on the internet.
|
||||||||||||||||||||||||||