Roboard Experience...
by PaulL
Mon Jul 05, 2010 10:26 pm
A progress update:
I have made significant progress in working on the Register classes. I found a few bugs in what I had before, so I have reworked things quite a bit. I have also made some significant deviations from the RoboIO source code at the I/O level.
For those unaware, much of the configuration of Roboard occurs through the PCI Configuration Registers at addresses CF8h and CFCh. Address CF8h refers to a specific configuration register, address CFCh refers to the data of the specified configuration register. These registers must be accessed as 32 bit registers. In the RoboIO code, a write is performed by setting CF8h as appropriate, reading 32 bits from CFCh, masking the intended bytes from the value of register CFCh, and adding the intended “Write” value when the value to be written is less than 32 bits. To sum it up, it performs WriteDWord (setting CF8h), ReadDword (reading CFCh), WriteDword (setting final CFCh value) for any North Bridge or South Bridge configuration registers less than 32 bits.
I have changed this entirely. ALL access to North Bridge or South Bridge now occur as 32 bit accesses. In short, I’ve taken a few 16 bit and 8 bit registers and grouped them into 32 bit registers. Since I already maintain the value of the register in the code, there is no need to re-read the Configuration Register’s value before writing it. The ONLY scenario where this would cause any sort of problem is if you have another program modifying the registers outside of this application at the same time (which isn’t a good idea anyway). So, the process for writing to the PCI Configuration Registers is now just WriteDWord (setting CF8h), WriteDWord (Setting CFCh).
As for accessing these “combined” registers, the functionality is broken out as separate functions in my classes, so this actually makes things simpler in the register and I/O code.
I have verified in several different ways that I can set configuration options, save them to an XML file, load them back from that XML file, and write them all back out to their proper registers. In short, I can configure the board’s configuration registers as I like, save them, and load them back, as easy as clicking a few buttons (literally, this is what I have in the app right now, see screenshot below).
So, setting the Servo (PWM) clock frequency, Servo Base Address, Servo Base Address Enable, and any other registers, is as simple as selecting or entering a few values in this configuration tool, then saving the configuration, and loading it later to use. Did I mention that I built this class such that I will be able to synchronize threaded calls to registers and functions? All I should have left to do is add a few Synclock blocks around a few sections of code.
Performance… Performance has been my primary concern from the start. Granted, this isn’t as fast as you’ll get from an unmanaged C++ application, but I can perform 8 bit writes in about 1.6 microseconds, and perform an 8 bit register read and retrieve a bit value from a function in about 1.9 microseconds (accessing the value without re-retrieving it from the physical register is about 0.06 microseconds). I’m using an unmanaged DLL to access RDTSC to get the CPU clock ticks to time these results, so they’re about as accurate as you can get, if not padded slightly by calling RDTSC itself. I even timed direct reads and writes, getting something like 400 nanoseconds overhead (time added for my register classes) on a write operation. I think, for the .Net framework, that’s not bad.
A 32 bit read off the North Bridge occurs in around 3.26 microseconds (slightly less than double the time for an 8 bit write to direct registers, but then that’s using two calls, one to write to CF8h, then one to read from CFCh).
I may be able to eek out a bit more performance, but I doubt it. The code is really clean once loaded, as most of the work is “front loaded” in order to keep running performance high. What I mean by this is that I’m setting as many values as I can when register and function objects are created so that those values don’t have to get recalculated later (costing more CPU time while running). Such values include value masks for register functions.
With this configuration tool, I’ve even managed to set the PWM / Servo parameters manually to move an attached servo. To do so programmatically will be a breeze. All I will need to do is set up the configuration, load it, then access the proper registers and make the proper changes to them to move servos (just changing HREG and LREG for each servo after the configuration is loaded). I can’t get it to be any easier than that!
The screenshot below is what things look like now. Highlighted values are value mappings that match Register values. Save XML saves the current Configuration to an XML file. Load XML loads the configuration into the application. Load Db loads the Configuration from a database (where I have a table that I use to do a clean load of the Configuration- Load DB is VERY VERY slow, and will not be used except for addition or removal of registers (such as to remove registers I simply don’t use for performance’s sake, or add registers I find later- using the Db is easier than editing the XML file directly). Read All reads all registers, starting with base address registers, then populating registers using the identified base address. Write All writes all register values back out, this will become the last step of LoadXML in code. Write is a test button for clocking performance, this was for a write to Chip Select for the External SPI Chip Select Register at the time I captured this image.
The tree can be manipulated live, meaning if you click to set a GPIO bit to On and it is in Output mode, you will output a 1 on the corresponding pin (verified with my DMM at 3.3 volts for High, 6.2 mV for low). The tree itself responds slowly (a second or so to update), and I am accessing it over Remote Desktop Connection, so that may add to it, but it’s a big tree, and it’s slow. For setting things up, it’s fine. The Treeview, or any Windows Forms UI for that matter, won’t be used in the running ‘bot app anyway. Another tidbit, the Register classes are separate from the UI stuff, so the Register classes are already standalone by design, the tree is just a way to visualize and manipulate what’s IN the register classes. 
Take Care,
Paul
I have made significant progress in working on the Register classes. I found a few bugs in what I had before, so I have reworked things quite a bit. I have also made some significant deviations from the RoboIO source code at the I/O level.
For those unaware, much of the configuration of Roboard occurs through the PCI Configuration Registers at addresses CF8h and CFCh. Address CF8h refers to a specific configuration register, address CFCh refers to the data of the specified configuration register. These registers must be accessed as 32 bit registers. In the RoboIO code, a write is performed by setting CF8h as appropriate, reading 32 bits from CFCh, masking the intended bytes from the value of register CFCh, and adding the intended “Write” value when the value to be written is less than 32 bits. To sum it up, it performs WriteDWord (setting CF8h), ReadDword (reading CFCh), WriteDword (setting final CFCh value) for any North Bridge or South Bridge configuration registers less than 32 bits.
I have changed this entirely. ALL access to North Bridge or South Bridge now occur as 32 bit accesses. In short, I’ve taken a few 16 bit and 8 bit registers and grouped them into 32 bit registers. Since I already maintain the value of the register in the code, there is no need to re-read the Configuration Register’s value before writing it. The ONLY scenario where this would cause any sort of problem is if you have another program modifying the registers outside of this application at the same time (which isn’t a good idea anyway). So, the process for writing to the PCI Configuration Registers is now just WriteDWord (setting CF8h), WriteDWord (Setting CFCh).
As for accessing these “combined” registers, the functionality is broken out as separate functions in my classes, so this actually makes things simpler in the register and I/O code.
I have verified in several different ways that I can set configuration options, save them to an XML file, load them back from that XML file, and write them all back out to their proper registers. In short, I can configure the board’s configuration registers as I like, save them, and load them back, as easy as clicking a few buttons (literally, this is what I have in the app right now, see screenshot below).
So, setting the Servo (PWM) clock frequency, Servo Base Address, Servo Base Address Enable, and any other registers, is as simple as selecting or entering a few values in this configuration tool, then saving the configuration, and loading it later to use. Did I mention that I built this class such that I will be able to synchronize threaded calls to registers and functions? All I should have left to do is add a few Synclock blocks around a few sections of code.
Performance… Performance has been my primary concern from the start. Granted, this isn’t as fast as you’ll get from an unmanaged C++ application, but I can perform 8 bit writes in about 1.6 microseconds, and perform an 8 bit register read and retrieve a bit value from a function in about 1.9 microseconds (accessing the value without re-retrieving it from the physical register is about 0.06 microseconds). I’m using an unmanaged DLL to access RDTSC to get the CPU clock ticks to time these results, so they’re about as accurate as you can get, if not padded slightly by calling RDTSC itself. I even timed direct reads and writes, getting something like 400 nanoseconds overhead (time added for my register classes) on a write operation. I think, for the .Net framework, that’s not bad.
A 32 bit read off the North Bridge occurs in around 3.26 microseconds (slightly less than double the time for an 8 bit write to direct registers, but then that’s using two calls, one to write to CF8h, then one to read from CFCh).
I may be able to eek out a bit more performance, but I doubt it. The code is really clean once loaded, as most of the work is “front loaded” in order to keep running performance high. What I mean by this is that I’m setting as many values as I can when register and function objects are created so that those values don’t have to get recalculated later (costing more CPU time while running). Such values include value masks for register functions.
With this configuration tool, I’ve even managed to set the PWM / Servo parameters manually to move an attached servo. To do so programmatically will be a breeze. All I will need to do is set up the configuration, load it, then access the proper registers and make the proper changes to them to move servos (just changing HREG and LREG for each servo after the configuration is loaded). I can’t get it to be any easier than that!
The screenshot below is what things look like now. Highlighted values are value mappings that match Register values. Save XML saves the current Configuration to an XML file. Load XML loads the configuration into the application. Load Db loads the Configuration from a database (where I have a table that I use to do a clean load of the Configuration- Load DB is VERY VERY slow, and will not be used except for addition or removal of registers (such as to remove registers I simply don’t use for performance’s sake, or add registers I find later- using the Db is easier than editing the XML file directly). Read All reads all registers, starting with base address registers, then populating registers using the identified base address. Write All writes all register values back out, this will become the last step of LoadXML in code. Write is a test button for clocking performance, this was for a write to Chip Select for the External SPI Chip Select Register at the time I captured this image.
The tree can be manipulated live, meaning if you click to set a GPIO bit to On and it is in Output mode, you will output a 1 on the corresponding pin (verified with my DMM at 3.3 volts for High, 6.2 mV for low). The tree itself responds slowly (a second or so to update), and I am accessing it over Remote Desktop Connection, so that may add to it, but it’s a big tree, and it’s slow. For setting things up, it’s fine. The Treeview, or any Windows Forms UI for that matter, won’t be used in the running ‘bot app anyway.
Another tidbit, the Register classes are separate from the UI stuff, so the Register classes are already standalone by design, the tree is just a way to visualize and manipulate what’s IN the register classes.
Take Care,
Paul
by PaulL
Wed Jul 07, 2010 7:32 am
Hey Roboard,
I agree in regard to the Flash SPI registers. I can attest to issues occuring, as when debugging, I managed to lock up my system and even set my hard drive to "not installed" in BIOS a few times, along with a few other "Wow" errors. I am lucky I didn't foul my board.
Do you know if anything else accesses the PCI Configuration registers while in Windows XP? I am about to post this tool with source code, and want to make sure some other aspect of Windows doesn't decide to write to the configuration registers in the middle of these classes writing to those same registers.
If I remember correctly, RoboIO.dll doesn't lock the CF8h / CFCh register read and write sequences into "real time priority", so if you haven't seen any issues under Windows XP, I shouldn't either.
Thanks,
Paul
I agree in regard to the Flash SPI registers. I can attest to issues occuring, as when debugging, I managed to lock up my system and even set my hard drive to "not installed" in BIOS a few times, along with a few other "Wow" errors. I am lucky I didn't foul my board.
Do you know if anything else accesses the PCI Configuration registers while in Windows XP? I am about to post this tool with source code, and want to make sure some other aspect of Windows doesn't decide to write to the configuration registers in the middle of these classes writing to those same registers.
If I remember correctly, RoboIO.dll doesn't lock the CF8h / CFCh register read and write sequences into "real time priority", so if you haven't seen any issues under Windows XP, I shouldn't either.
Thanks,
Paul
by roboard
Wed Jul 07, 2010 10:38 am
RN1AsOf091407 wrote:Do you know if anything else accesses the PCI Configuration registers while in Windows XP? I am about to post this tool with source code, and want to make sure some other aspect of Windows doesn't decide to write to the configuration registers in the middle of these classes writing to those same registers.
Hi, Paul,
in general, only hardware drivers may access 0xcfc & 0xcf8 registers. And RoBoard's XP drivers basically don't access the two registers when RoBoIO is running. So we haven't met any issue about this under WinXP.
by PaulL
Sat Jul 10, 2010 2:46 am
Hi Roboard,
I do know that for acquiring servo position, calls to the PCI Configuration Register occur to change setting of pins from PWM to GPIO-Input to pick up the position feedback pulse, then back to PWM.
At some point, I may write a driver if I encounter issues, but for now, I'm going to provide the code as-is, with the same methods used in RoboIO.
Thanks,
Paul
I do know that for acquiring servo position, calls to the PCI Configuration Register occur to change setting of pins from PWM to GPIO-Input to pick up the position feedback pulse, then back to PWM.
At some point, I may write a driver if I encounter issues, but for now, I'm going to provide the code as-is, with the same methods used in RoboIO.
Thanks,
Paul
Accel / Decel .Net Code...
by PaulL
Sat Jul 17, 2010 3:29 am
See below, this is beta code for servo moves. In short, a timer event will call GetMoveValue as appropriate for each servo to get the appropriate position based on the "FractionOfMove". FractionOfMove is 0 to 1, 0 meaning the move just started, 1 meaning move is complete, anything between 0 and 1 being some fraction of the actual move. This means that .5 is halfway through the move, for example.
Note, this is JUST move code, and doesn't represent servos, nor hardware, just the code to calculate positions for moves.
Move Methods are in the Enum, but detailed here:
* AccelDecel - Accelerate from a starting position, then decelerate to some end position.
* Accel - Accelerate and abruptly stop at full acceleration (like a punch or kick or jump).
* Decel - Decelerate from a rapid speed then slowly stop (like catching yourself from a fall)
* Linear - Move from point a to point b with no acceleration or deceleration, like other stock servo moves in robots I've seen.
The intention is to chain together sequences of move types to generate motion with nuances, as well as to calculate using changes in end position or move duration. Changes to end position or move duration will likely invoke a new move instance. One function not yet built is to transition from one acceleration value to another. I may simply force changes to duration or end point to occur slowly so as not to "jerk" during a move that has its duration or end point changed. Probably will end up with something like a "smoothing factor" to control how quickly end point or duration is transitioned to the new desired value.
I've run this code and generated some nice curves in Excel with the results.
Take Care,
Paul
Note, this is JUST move code, and doesn't represent servos, nor hardware, just the code to calculate positions for moves.
Move Methods are in the Enum, but detailed here:
* AccelDecel - Accelerate from a starting position, then decelerate to some end position.
* Accel - Accelerate and abruptly stop at full acceleration (like a punch or kick or jump).
* Decel - Decelerate from a rapid speed then slowly stop (like catching yourself from a fall)
* Linear - Move from point a to point b with no acceleration or deceleration, like other stock servo moves in robots I've seen.
The intention is to chain together sequences of move types to generate motion with nuances, as well as to calculate using changes in end position or move duration. Changes to end position or move duration will likely invoke a new move instance. One function not yet built is to transition from one acceleration value to another. I may simply force changes to duration or end point to occur slowly so as not to "jerk" during a move that has its duration or end point changed. Probably will end up with something like a "smoothing factor" to control how quickly end point or duration is transitioned to the new desired value.
I've run this code and generated some nice curves in Excel with the results.
Take Care,
Paul
- Code: Select all
Enum MoveTypes
AccelDecel
Accel
Decel
Linear
End Enum
Function GetMoveValue(ByVal MvType As MoveTypes, ByVal StartPos As Integer, ByVal EndPos As Integer, ByVal FractionOfMove As Double) As Double
Dim Offset As Integer
Dim Distance As Integer
If StartPos > EndPos Then
Distance = StartPos - EndPos
Offset = EndPos
Select Case MvType
Case MoveTypes.Accel
Return Distance - CalculatePos_Accel(Distance, FractionOfMove) + Offset
Case MoveTypes.AccelDecel
Return Distance - CalculatePos_AccelDecel(Distance, FractionOfMove) + Offset
Case MoveTypes.Decel
Return Distance - CalculatePos_Decel(Distance, FractionOfMove) + Offset
Case MoveTypes.Linear
Return Distance - CalculatePos_Linear(Distance, FractionOfMove) + Offset
End Select
Else
Distance = EndPos - StartPos
Offset = StartPos
Select Case MvType
Case MoveTypes.Accel
Return CalculatePos_Accel(Distance, FractionOfMove) + Offset
Case MoveTypes.AccelDecel
Return CalculatePos_AccelDecel(Distance, FractionOfMove) + Offset
Case MoveTypes.Decel
Return CalculatePos_Decel(Distance, FractionOfMove) + Offset
Case MoveTypes.Linear
Return CalculatePos_Linear(Distance, FractionOfMove) + Offset
End Select
End If
End Function
Function CalculatePos_AccelDecel(ByVal Distance As Integer, ByVal FractionOfMove As Double) As Double
If FractionOfMove < 0.5 Then 'Accel:
Return CalculatePos_Accel(Distance * 0.5, FractionOfMove * 2)
ElseIf FractionOfMove > 0.5 Then 'Decel:
Return CalculatePos_Decel(Distance * 0.5, (FractionOfMove - 0.5) * 2) + (Distance * 0.5)
Else '= .5! Exact Middle.
Return Distance / 2
End If
End Function
Function CalculatePos_Accel(ByVal Distance As Integer, ByVal FractionOfMove As Double) As Double
Return FractionOfMove * (Distance * FractionOfMove)
End Function
Function CalculatePos_Decel(ByVal Distance As Integer, ByVal FractionOfMove As Double) As Double
FractionOfMove = 1 - FractionOfMove
Return Distance - (FractionOfMove * (Distance * FractionOfMove))
End Function
Function CalculatePos_Linear(ByVal Distance As Integer, ByVal FractionOfMove As Double) As Double
Return (Distance * FractionOfMove)
End Function
Graphed Results of Motion Code...
by PaulL
Sun Jul 18, 2010 2:02 pm
I have been working on sequencing moves from the RN-1 for Roboard, below is an example of the algorithms doing what they do across 3 repeating moves. Starting at 1500, to 1658, to 1605, back to 1500, and repeating. These are actual servo move values, scaled from RoboBasic moves. Notice the steepness of the transition from 1605 to 1500 for the accel / decel move (blue line, 3rd move), this is set to half the time of the other moves. With all moves being equal duration, the curves are very fluid. Even this example isn't too bad. In typical servo motion such as in the MRC-3024's stock program, you get the line drawn in red, no acceleration or deceleration. All moves are using accel / decel in this example for the accel / decel line. The Linear moves were also generated by this same code, selecting the "Linear" mode for the move.
With my RN-1, much of the instability in his stock configuration (oscillating motion after a move has stopped) is caused by sudden starts and stops of servo motion, you get lots of "jerkiness" with that approach. It's like stepping on the gas in your car, and suddenly being at 25 mph, then stopping from 25 mph to zero suddenly when you hit the brake. If you were to do that over 20 seconds, imagine accelerating to 50 mph and decelerating to 50 mph in that same amount of time versus sudden starting and stopping. This is an approximate comparison to what I'm doing here.
I can't wait to see him walk with this code, it should be quite different. A further note, moves can presently end early if their duration is less than the duration for other moves in the intended "frame".
As you can tell, I'm very close to having my RN1 walking again!
Take Care,
Paul
With my RN-1, much of the instability in his stock configuration (oscillating motion after a move has stopped) is caused by sudden starts and stops of servo motion, you get lots of "jerkiness" with that approach. It's like stepping on the gas in your car, and suddenly being at 25 mph, then stopping from 25 mph to zero suddenly when you hit the brake. If you were to do that over 20 seconds, imagine accelerating to 50 mph and decelerating to 50 mph in that same amount of time versus sudden starting and stopping. This is an approximate comparison to what I'm doing here.
I can't wait to see him walk with this code, it should be quite different.
A further note, moves can presently end early if their duration is less than the duration for other moves in the intended "frame".
As you can tell, I'm very close to having my RN1 walking again!
Take Care,
Paul
Servos Moving Nicely via VB.Net...
by PaulL
Sun Jul 18, 2010 11:28 pm
ANOTHER SUCCESS!!!
I have 3 servos connected (much more, and I'll probably kick out my little benchtop power supply), running acceleration / deceleration calculations for all positions across three moves, and the CPU usage doesn't nudge above the 2% it averages with remote desktop screen updates for the CPU meter (sitting still, no code running). I'm using 4mS updates, meaning my position update timer fires every 4mS to update all servo positions (using a wrapper around the Multimedia timer, one of the better timers in Windows, with RDTSC used to calculate time elapsed).
A quick note, using a timer alone would probably not be the best way to do position updates. In my work thus far, I have learned that you simply can't count on Windows timers to accurately reflect time elapsed.
I can drag and drop the app window while the servos are moving, and the CPU usage is much higher just to move the window, and yet doesn't interfere at all with the servo position updates. No stumbling, no fumbling, just nice, clean motion. This is really lean code for VB.Net!
Now, I need to code some trims into the mix, get his power straightened out, properly cable the HSR-5498SG's, map the servo code to the correct RN1 servos based on where I plug them in, adjust trims, and I can get him walking!!!
As I type this, I tried for one more servo, and kicked out my power supply. Oh, the joys of insufficient amperage... Need more POWER!
I'm pretty happy with the register code and the servo move code (though, I will likely need to modify load sequence values in the Register Config db for other functionality to set configuration registers for other functions as I build those). I will probably rework the move sequencing aspect into something more flexible, but I can do simple frame-based motion as in the stock MRC-3024 for now. With acceleration and deceleration added....
Once he's back walking, I want to rework the "wing move" sequence, I think I can get some really graceful flying motion.
Take Care,
Paul
I have 3 servos connected (much more, and I'll probably kick out my little benchtop power supply), running acceleration / deceleration calculations for all positions across three moves, and the CPU usage doesn't nudge above the 2% it averages with remote desktop screen updates for the CPU meter (sitting still, no code running). I'm using 4mS updates, meaning my position update timer fires every 4mS to update all servo positions (using a wrapper around the Multimedia timer, one of the better timers in Windows, with RDTSC used to calculate time elapsed).
A quick note, using a timer alone would probably not be the best way to do position updates. In my work thus far, I have learned that you simply can't count on Windows timers to accurately reflect time elapsed.
I can drag and drop the app window while the servos are moving, and the CPU usage is much higher just to move the window, and yet doesn't interfere at all with the servo position updates. No stumbling, no fumbling, just nice, clean motion. This is really lean code for VB.Net!
Now, I need to code some trims into the mix, get his power straightened out, properly cable the HSR-5498SG's, map the servo code to the correct RN1 servos based on where I plug them in, adjust trims, and I can get him walking!!!
As I type this, I tried for one more servo, and kicked out my power supply. Oh, the joys of insufficient amperage... Need more POWER!
I'm pretty happy with the register code and the servo move code (though, I will likely need to modify load sequence values in the Register Config db for other functionality to set configuration registers for other functions as I build those). I will probably rework the move sequencing aspect into something more flexible, but I can do simple frame-based motion as in the stock MRC-3024 for now. With acceleration and deceleration added....
Once he's back walking, I want to rework the "wing move" sequence, I think I can get some really graceful flying motion.
Take Care,
Paul
servos and .net
by TheWaldo
Mon Jul 19, 2010 10:29 pm
Can you post your updated library for this.. I am about to start a project that uses the Roboard servo controller from a .net(C#) program..
by PaulL
Sat Jul 24, 2010 6:40 pm
Update:
It will take some time to get the Hitec remote protocol and subsequent receiver code sorted out, so for now I need a quick way to command my robot.
I will need a command mode for it no matter what, and I intended to do this anyway, so here we are: voice recognition.
I have managed to find (for some reason with some difficulty) examples of utilizing the speech recognition functions of SpeechLib. It's not so much that I just want to do speech recognition, I want to create a bot-specific vocabulary so the speech engine doesn't mangle words into ugly unintended sentences (which it is quite good at!). I have found what I was looking for!!!
I have a means to set a vocabulary for the robot, so I can use the microphone input on Roboard for speech recognition. With a few commands so far, the recognition results look very promising.
One aspect I have concern about is whether or not he will be able to "listen" while his servos are moving. I think this will entail a bit of creativity in microphone positioning to isolate servo sounds from the direction the microphone is pointed in. I guess it's not too much of a problem to have him stop to listen, but it's not what I would prefer. Only practical application will tell me if I can get him to listen while walking, for example. I think he will have a much better chance with a limited command set. By limited, I mean perhaps a few hundred words or phrases instead of thousands.
My first go at Speech Recognition will simply be to recall the Robonova-1 move sequences and execute them on Roboard. I am close to that.
It (my efforts in Roboard development) is actually starting to get very interesting now that I have the basic register access and servo move stuff out of the way.
I am thinking I may implement a "talk back" mode to verify input before executing a command for the time being (until I'm satisfied with / confident in the speech recognition results).
I don't believe my Roboard will be happy with a speaker strapped directly to its audio output connector, so I need a very tiny 1 to 2 watt audio amplifier, preferably decent quality, capable of driving a 4 ohm speaker (I plan to rob an old laptop of its speakers). Any ideas? Possibly stereo if I implement a "walking MP3 player" mode...
Take Care,
Paul
It will take some time to get the Hitec remote protocol and subsequent receiver code sorted out, so for now I need a quick way to command my robot.
I will need a command mode for it no matter what, and I intended to do this anyway, so here we are: voice recognition.
I have managed to find (for some reason with some difficulty) examples of utilizing the speech recognition functions of SpeechLib. It's not so much that I just want to do speech recognition, I want to create a bot-specific vocabulary so the speech engine doesn't mangle words into ugly unintended sentences (which it is quite good at!). I have found what I was looking for!!!
I have a means to set a vocabulary for the robot, so I can use the microphone input on Roboard for speech recognition. With a few commands so far, the recognition results look very promising.
One aspect I have concern about is whether or not he will be able to "listen" while his servos are moving. I think this will entail a bit of creativity in microphone positioning to isolate servo sounds from the direction the microphone is pointed in. I guess it's not too much of a problem to have him stop to listen, but it's not what I would prefer. Only practical application will tell me if I can get him to listen while walking, for example. I think he will have a much better chance with a limited command set. By limited, I mean perhaps a few hundred words or phrases instead of thousands.
My first go at Speech Recognition will simply be to recall the Robonova-1 move sequences and execute them on Roboard. I am close to that.
It (my efforts in Roboard development) is actually starting to get very interesting now that I have the basic register access and servo move stuff out of the way.
I am thinking I may implement a "talk back" mode to verify input before executing a command for the time being (until I'm satisfied with / confident in the speech recognition results).
I don't believe my Roboard will be happy with a speaker strapped directly to its audio output connector, so I need a very tiny 1 to 2 watt audio amplifier, preferably decent quality, capable of driving a 4 ohm speaker (I plan to rob an old laptop of its speakers). Any ideas? Possibly stereo if I implement a "walking MP3 player" mode...
Take Care,
Paul
by matt.stevenson
Sun Jul 25, 2010 12:55 am
For speech recognition, in the past, I have used a Sensory Fluent. It allows for small limited vocabularies. It boasts up to 50,000 words but I don't trust that number. Their main market is embedded chips for speech rec but they do have an SDK. It doesn't look like its free so I'm not sure how my employer got their hands on it. We were using it on a laptop, but it was supposed to be pretty effecient (I never did runtime analysis though). It sounds like you have SpeechLib running pretty well though, but it could be something to look at.
There is also Nuance which produced Dragon Naturally Speaking, but it is not free either, so you may be best just sticking to SpeechLib.
As for cleaning up the background noise, I've been interested in Audience. They make chips for cell phones using two mics to filter out noise. I'm not sure if it is terribly useful outside of cellphones, but I have been interested in what they can do.
I know a researcher who is working on sound localization and noise filtering using a small 4 mic array (mems mics hooked to an ADC). He's said he should have something working by the end of the summer, so depending on how that goes I might be able to send you some code later.
For a talk back mode, I have been actively working on that for my roboard. I chose to go with FreeTTS, an open source Java Text-to-Speech library. The default voice is not very good, but it runs fast (takes about 10-15% of cpu when speaking if I remember correctly). There are some additional voice packs offered by CMU (I like the rab voice), but I have so far been unsuccessful at compiling the voice into a useable jar.
I have been using the TTS with a different project (using the roboard) from the robots I work on, so I've had the luxury of using headphones or a computer speaker set. I would be interested to know what you end up using for speakers.
I also want to implement an MP3 player. I'm keeping as much of my project in Java as possible and it looks like you're using VB so I'm not sure how much we can collaborate on an MP3 player. Although it shouldn't be hard to get something simple working in any language.
There is also Nuance which produced Dragon Naturally Speaking, but it is not free either, so you may be best just sticking to SpeechLib.
As for cleaning up the background noise, I've been interested in Audience. They make chips for cell phones using two mics to filter out noise. I'm not sure if it is terribly useful outside of cellphones, but I have been interested in what they can do.
I know a researcher who is working on sound localization and noise filtering using a small 4 mic array (mems mics hooked to an ADC). He's said he should have something working by the end of the summer, so depending on how that goes I might be able to send you some code later.
For a talk back mode, I have been actively working on that for my roboard. I chose to go with FreeTTS, an open source Java Text-to-Speech library. The default voice is not very good, but it runs fast (takes about 10-15% of cpu when speaking if I remember correctly). There are some additional voice packs offered by CMU (I like the rab voice), but I have so far been unsuccessful at compiling the voice into a useable jar.
I have been using the TTS with a different project (using the roboard) from the robots I work on, so I've had the luxury of using headphones or a computer speaker set. I would be interested to know what you end up using for speakers.
I also want to implement an MP3 player. I'm keeping as much of my project in Java as possible and it looks like you're using VB so I'm not sure how much we can collaborate on an MP3 player. Although it shouldn't be hard to get something simple working in any language.
by PaulL
Sun Jul 25, 2010 3:36 am
UGH! I hate when I write up a post and it goes poof on Submit. Ugh!
Matt,
Thanks for the feedback and info, I will check into it further (Had a longer reply written up, but lost it when "Submit" went south on me). I'll try again tomorrow, but frustrated at the moment to write a lot, then watch it all vaporize into a lost request in cyberspace... I usually copy my post before I submit, but ran out of luck when I neglected to do so tonight.
General Update:
I ended up going to Radio Shack to attempt to find Header pins (simple request, right?)- no such luck. I was going to build a splicer to split the signals and power so I could run the power from somewhere other than through the Roboard, but not possible without cutting wires. So, I'm moving my Roboard to my almost stock RN-1 to connect the 16 HSR-8498HB servos directly. I'll power with either new LiPo, tethered 7.2v, or stock battery.
The Speech wrapper I've built works well, very little CPU utilization, very simple to use, reads commands in from a text file (one command - word or phrase - per line in the file) upon initialization. It's pretty accurate, and is working better than expected.
Still hoping on walking by the end of the weekend and posting src code if all goes well.
Take Care,
Paul
Matt,
Thanks for the feedback and info, I will check into it further (Had a longer reply written up, but lost it when "Submit" went south on me). I'll try again tomorrow, but frustrated at the moment to write a lot, then watch it all vaporize into a lost request in cyberspace... I usually copy my post before I submit, but ran out of luck when I neglected to do so tonight.
General Update:
I ended up going to Radio Shack to attempt to find Header pins (simple request, right?)- no such luck. I was going to build a splicer to split the signals and power so I could run the power from somewhere other than through the Roboard, but not possible without cutting wires. So, I'm moving my Roboard to my almost stock RN-1 to connect the 16 HSR-8498HB servos directly. I'll power with either new LiPo, tethered 7.2v, or stock battery.
The Speech wrapper I've built works well, very little CPU utilization, very simple to use, reads commands in from a text file (one command - word or phrase - per line in the file) upon initialization. It's pretty accurate, and is working better than expected.
Still hoping on walking by the end of the weekend and posting src code if all goes well.
Take Care,
Paul
by PaulL
Sun Aug 08, 2010 1:15 am
I've been tied up with issues related to sound, but last weekend, I did manage to start working on the SPI / ADC code. It's not done yet, but I have been working on it.
I'd really like to find a solution to my sound problem, but I just don't get it- why every 8 or 9 seconds is there a spike in CPU when either using Speech, or recording with Sound Recorder?
I would really like to know, Linux or Windows, if anyone has watched CPU utilization when recording...
Further, a different sound card has another advantage - stereo microphones. It should be pretty straightforward to compare left and right levels to determine sound position relative to my 'bot...
If I can't have him do reliable and non-CPU-intensive voice recognition and speech, there really isn't much point in continuing. I'm already frustrated with whatever Hitec is or isn't doing to cause me so much trouble in finding replacement gears for the HSR5498SG's.
What frustrates me is that my desktop machine, though much different than Roboard, doesn't notice (CPU utilization-wise) that I turn on and use Voice Recognition and Speech. It's not an exceptional machine, not even "current" by some standards. Roboard, however, every 8 seconds or so, maxes up to 100% CPU, and voice playback cuts out due to the CPU usage.
If someone manufactured a Mini PCI sound card that worked with the Vortex86DX in Roboard, I'd buy it right now.
Paul
I'd really like to find a solution to my sound problem, but I just don't get it- why every 8 or 9 seconds is there a spike in CPU when either using Speech, or recording with Sound Recorder?
I would really like to know, Linux or Windows, if anyone has watched CPU utilization when recording...
Further, a different sound card has another advantage - stereo microphones. It should be pretty straightforward to compare left and right levels to determine sound position relative to my 'bot...
If I can't have him do reliable and non-CPU-intensive voice recognition and speech, there really isn't much point in continuing. I'm already frustrated with whatever Hitec is or isn't doing to cause me so much trouble in finding replacement gears for the HSR5498SG's.
What frustrates me is that my desktop machine, though much different than Roboard, doesn't notice (CPU utilization-wise) that I turn on and use Voice Recognition and Speech. It's not an exceptional machine, not even "current" by some standards. Roboard, however, every 8 seconds or so, maxes up to 100% CPU, and voice playback cuts out due to the CPU usage.
If someone manufactured a Mini PCI sound card that worked with the Vortex86DX in Roboard, I'd buy it right now.
Paul