DE-10 nano temperature issues

https://github.com/MiSTer-devel/Main_MiSTer/wiki

Moderators: Mug UK, Zorro 2, Greenious, spiny, Sorgelig, Moderator Team

ijor
Hardware Guru
Hardware Guru
Posts: 3293
Joined: Sat May 29, 2004 7:52 pm
Contact:

DE-10 nano temperature issues

Postby ijor » Fri Nov 03, 2017 1:34 am

I finally had the chance to perform some temperature tests with similar boards. There is definitely a problem with the DE-10 nano board. It is not the FPGA chip, or at least it is not just the chip. Other Terasic boards with the same or similar FPGA get worm, but the nano gets really hot.

For comparison I tested the DE-10 Standard. It's the big brother of the DE-10 Nano. It has a variation of the same FPGA. But it comes with much more peripherals including on board SDRAM. I run the default, factory built, Linux configuration. The FPGA chip warms to about 37 degrees Celsius.

The DE-10 nano on the other hand, also running the default Linux build and without SDRAM board, it gets much warmer. Initially the FPGA runs at about the same temperature, 37 degrees C. But after a few minutes temperature starts to rise and can approximate the 50 degrees C! Running actual MiSTer cores go a bit warmer. The mem test at 140 MHz runs at about 53 degrees C. And the ao486 can reach something like 57 degrees C.

While getting close to 60 degrees C is a lot for this kind of board, it is still well in the recommended operating range. Cores should not fail if they meet timing with the slow model. It makes you think that perhaps some cores are not properly constrained.

Btw, I didn't note before, but the Nano comes with an industrial grade FPGA chip that has an extended temperature range (up to 100 degrees C!). That's quite striking. All the other Terasic Cyclone boards I've seen come with the much more common and cheaper commercial grade FPGA. Why Terasic decided to use a more expensive chip precisely in the cheapest board? Were they aware about potential heating issues in the nano?

The tests were performed at a room temperature of about 25 degrees C. Without any kind of heatsink or fan.

Btw, I have to say that the work done by Sorgelig is impressive. The setup guide by gadagatika is also awesome!

Sorgelig
Atari God
Atari God
Posts: 1933
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: DE-10 nano temperature issues

Postby Sorgelig » Fri Nov 03, 2017 1:47 am

DE10-nano is smaller than DE10-standard where many hot chips are close together. For example DDR memory also produce noticeable heat which just warm up the FPGA more. Also surrounded tall Arduino connectors make airflow worse and also add more heat.
May be that's why Terasic uses industrial grade FPGA.
As i've mentioned earlier, heatsink with at least slow rotating FAN is a must have addon for MiSTer. Some cores like ao486 impossible to tweak to make it work reliably with any temperature of FPGA. Core is too big and even without much constrains took more than hour to compile.

ijor
Hardware Guru
Hardware Guru
Posts: 3293
Joined: Sat May 29, 2004 7:52 pm
Contact:

Re: DE-10 nano temperature issues

Postby ijor » Sun Nov 05, 2017 8:58 pm

Sorgelig wrote:DE10-nano is smaller than DE10-standard where many hot chips are close together. For example DDR memory also produce noticeable heat which just warm up the FPGA more. Also surrounded tall Arduino connectors make airflow worse and also add more heat.
May be that's why Terasic uses industrial grade FPGA.

As i've mentioned earlier, heatsink with at least slow rotating FAN is a must have addon for MiSTer.


Yes, the mechanical design is probably one of the reasons of the heating of the nano. But if Terasic was aware, then it is strange they didn't make it slightly longer. Assuming that is the only problem. In some other place there is a mention of design of the power logic as well.

It is interesting to note that in small quantities, the industrial grade part used in the nano costs about $250 !!! That's about twice the retail price of the whole board, let alone the discount price for students. Yeah, I realize they don't pay that price for the part, still ...

Some cores like ao486 impossible to tweak to make it work reliably with any temperature of FPGA. Core is too big and even without much constrains took more than hour to compile.


But the ao486 is probably an extreme case. Most, if not all, the other cores probably fit comfortably and they could be constrained properly (assuming they are not already). And I am not complaining to you, that should be implemented by the original developers that are very familiar with the design.

I was looking at the SOC power management. The hardware does have some features but seem they are not enabled by the (default Altera) software. Unfortunately that kind of tweaking is quite complicated. Doing changes that are more appropriate to bare metal development on top of the full Linux OS is not easy. Especially when you don't know beforehand if its worth the effort.

bernouilli
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 100
Joined: Fri Jan 06, 2017 5:02 pm
Location: Paris/France

Re: DE-10 nano temperature issues

Postby bernouilli » Sun Nov 05, 2017 10:03 pm

I'm too looking for what can be done to reduce the temperature. I found some interesting information about power management for the cyclone V.
I know that the MiSTer program takes a lot of cpu but even if you don't run this program, the de10 gets pretty hot (over 50° without any heatsink).
I think that the first thing to do is trying to run the mpu at a lower frequency. I don't think it's necessary to run at 800mhz. 400 or even 200mhz should be enough.
There is a register that handles this but it cannot be used within the MiSTer program. The frequency must be set when building the preloader.
Right now I just managed to setup a compilation chain for building the MiSTer executable. I think I also need to install the quartus suite to start modifying the linux part.
Then there are other paths to follow like the WFE or/and WFI states.

ijor
Hardware Guru
Hardware Guru
Posts: 3293
Joined: Sat May 29, 2004 7:52 pm
Contact:

Re: DE-10 nano temperature issues

Postby ijor » Mon Nov 06, 2017 8:15 pm

bernouilli wrote:I think that the first thing to do is trying to run the mpu at a lower frequency. I don't think it's necessary to run at 800mhz. 400 or even 200mhz should be enough.


I agree. It might be interesting also to test one vs two cores. Power consumption and heating is not proportional to the frequency. So in some cases two cores at 200 MHz might be more efficient than one at 400 (or two at 100 MHz instead of one at 200 MHz).

Then there are other paths to follow like the WFE or/and WFI states.


Yep, that would be ideal. But this might require a new API with the FPGA side, otherwise the CPU will never be idle.

The hardware also supports automatically powering down the DDR when idle. Seems this is not enabled for some reason. But for this to be really efficient the 512K L2 cache should be enough both for the app and the kernel. It is also possible to reduce the DDR frequency as well, at least in theory.

Sorgelig
Atari God
Atari God
Posts: 1933
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: DE-10 nano temperature issues

Postby Sorgelig » Tue Nov 07, 2017 2:25 am

DDR is used for video scaler and it's quite busy in all cores.

ijor
Hardware Guru
Hardware Guru
Posts: 3293
Joined: Sat May 29, 2004 7:52 pm
Contact:

Re: DE-10 nano temperature issues

Postby ijor » Tue Nov 07, 2017 1:13 pm

Sorgelig wrote:DDR is used for video scaler and it's quite busy in all cores.


Yes, of course, forgot about that.

Btw, Terasic documentation mentions that there are three different versions of the nano. Well, actually four, A, B, B2 and C. Revision C is mentioned to use different power components. I wonder if it might have some impact on the heating. Somebody happen to have a revision C board? I bought the nano from a major Terasic distributor just a couple of weeks ago, but oddly enough is revision B, not even revision B2.

Sorgelig
Atari God
Atari God
Posts: 1933
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: DE-10 nano temperature issues

Postby Sorgelig » Tue Nov 07, 2017 1:54 pm

ijor wrote:Btw, Terasic documentation mentions that there are three different versions of the nano. Well, actually four, A, B, B2 and C. Revision C is mentioned to use different power components. I wonder if it might have some impact on the heating. Somebody happen to have a revision C board? I bought the nano from a major Terasic distributor just a couple of weeks ago, but oddly enough is revision B, not even revision B2.

It's about the components on nano board - it has no relation to ARM Power Management inside FPGA chip.

User avatar
Newsdee
Atari God
Atari God
Posts: 1129
Joined: Fri Sep 19, 2014 8:40 am

Re: DE-10 nano temperature issues

Postby Newsdee » Tue Nov 07, 2017 2:34 pm

How do we find out board revision?

Sorgelig
Atari God
Atari God
Posts: 1933
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: DE-10 nano temperature issues

Postby Sorgelig » Tue Nov 07, 2017 2:38 pm

Newsdee wrote:How do we find out board revision?

it's on bottom side of the board. Under GPIO0 connector. You can see the number made on copper layer.
something like 10-01610140-B0.
-B0 means B0 revision.

ijor
Hardware Guru
Hardware Guru
Posts: 3293
Joined: Sat May 29, 2004 7:52 pm
Contact:

Re: DE-10 nano temperature issues

Postby ijor » Tue Nov 07, 2017 2:51 pm

Sorgelig wrote:It's about the components on nano board - it has no relation to ARM Power Management inside FPGA chip.


I know of course. I didn't mean that those changes would affect the power management. But since there are some claims that the heating might be related to the power logic, I am wondering if the newer revision might be better in this regard.

Sorgelig
Atari God
Atari God
Posts: 1933
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: DE-10 nano temperature issues

Postby Sorgelig » Tue Nov 07, 2017 5:04 pm

ijor wrote:But since there are some claims that the heating might be related to the power logic, I am wondering if the newer revision might be better in this regard.

it's related to power logic, yes. Power logic INSIDE FPGA. ARM CPU in Cyclone V has no power management. It cannot sleep/idle. So, it's absolutely doesn't matter if MiSTer binary takes 100% or 0% of CPU time - it won't affect the heating.

User avatar
alexh
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2651
Joined: Wed Oct 20, 2004 1:52 pm
Location: UK - Oxford
Contact:

Re: DE-10 nano temperature issues

Postby alexh » Tue Nov 07, 2017 5:14 pm

Sorgelig wrote:ARM CPU in Cyclone V has no power management. It cannot sleep/idle. So, it's absolutely doesn't matter if MiSTer binary takes 100% or 0% of CPU time - it won't affect the heating.

Are you sure?

The design document for the Cyclone V HPS seems to say otherwise :

https://www.altera.com/en_US/pdfs/liter ... /an734.pdf

ijor
Hardware Guru
Hardware Guru
Posts: 3293
Joined: Sat May 29, 2004 7:52 pm
Contact:

Re: DE-10 nano temperature issues

Postby ijor » Tue Nov 07, 2017 5:28 pm

Sorgelig wrote:it's related to power logic, yes. Power logic INSIDE FPGA...


No. Not the power logic INSIDE the FPGA. As I said at the start of this thread, I tested other Terasic boards with the same FPGA. The same Cyclone V SOC with the same power management. Yet they don't produce, not nearly, the heat that you can see at the DE-10 nano. There is something specific, that must be board level, and not inside the FPGA, that affects the nano and not the others.

I agree that the mechanical design, the proximity of the components is probably one of the reasons. But we don't know that for sure. And certainly we don't know if that is the only reason. As I commented in other thread, there are reports that the problem is related to the board level power logic design. I, of course, can't say if that is accurate or not.

Sorgelig
Atari God
Atari God
Posts: 1933
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: DE-10 nano temperature issues

Postby Sorgelig » Tue Nov 07, 2017 5:31 pm

ijor wrote:But we don't know that for sure.

speak for your self ;)
I'm sure.

Sorgelig
Atari God
Atari God
Posts: 1933
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: DE-10 nano temperature issues

Postby Sorgelig » Tue Nov 07, 2017 5:43 pm

alexh wrote:Are you sure?

The design document for the Cyclone V HPS seems to say otherwise :

https://www.altera.com/en_US/pdfs/liter ... /an734.pdf

Where it says otherwise? Where it tells about cpu idling or dynamic frequency change like cpu in phone for example? There is NO power management in its classic form. Document only describes how to reduce power by switching off unneeded components. It's applicable only for bare-metal programming, not for Linux.
Instead pointing me to documents, i suggest to build your own version of Linux where you will greatly reduce the heating. Everything is open source, so just take it and improve if you think you know it better.

ijor
Hardware Guru
Hardware Guru
Posts: 3293
Joined: Sat May 29, 2004 7:52 pm
Contact:

Re: DE-10 nano temperature issues

Postby ijor » Tue Nov 07, 2017 7:12 pm

Sorgelig wrote:speak for your self ;) I'm sure.


Sorry, I honestly don't see how can you be so sure. At least about it being the only reason.

Not too long ago you refused to accept that there was a heating problem on the DE-10 Nano at all. You were sure that it was generic to the SOC and that any board using the same FPGA should produce the same heating. Furthermore, you were so sure that you said that anybody claiming otherwise (and I quote) "is most likely competitors envy".

So now that you have no choice but to accept that there is a specific issue on this board, you suddenly are absolutely sure about the reason and that is the only reason at all. Well, you might be sure. That doesn't mean you are right.

Instead pointing me to documents, i suggest to build your own version of Linux where you will greatly reduce the heating. Everything is open source, so just take it and improve if you think you know it better.


Man, please stop being so aggressive. You complain all the time that it is too much work for a single developer. And you are absolutely right about that. You made an awesome job and mostly alone. But you are not helping yourself like that. Being arrogant and aggressive almost at the point of making a fight for any voice of disagreement is, IMHO, not the best way.

User avatar
Newsdee
Atari God
Atari God
Posts: 1129
Joined: Fri Sep 19, 2014 8:40 am

Re: DE-10 nano temperature issues

Postby Newsdee » Wed Nov 08, 2017 12:04 am

Other than heat itself, what is the consequence for this board and cores? I think ao486 becomes unstable, but other than that? In other words, how much does it matter for the "end user"?

It's definitely worth it to experiment and find out the causes. It benefits not only MiSTer but also anybody using the DE10 Nano. And as we go forward we might find other problems with the board. Those might be solvable with big or small changes...but I don't think anybody is expecting the changes or the experiments to be done right away, right? Let's just be transparent and work things through by documenting and testing things collaboratively :)

Sorgelig
Atari God
Atari God
Posts: 1933
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: DE-10 nano temperature issues

Postby Sorgelig » Wed Nov 08, 2017 4:48 am

ijor wrote:Not too long ago you refused to accept that there was a heating problem on the DE-10 Nano at all. You were sure that it was generic to the SOC and that any board using the same FPGA should produce the same heating.

I never said the heating is not a problem in general. It's not a problem for most cores. Actually for all cores except ao486. So, if you don't use ao486 then you can disregard the cooler recommendation. After porting the ao486 i found that every iteration of compilation this core takes 1-2hours(!) i found that cooling the chip is a better alternative. So i've released a new recommendations together with new I/O board which accommodates the fan. It doesn't contradict to my previous words. And i'm still saying that every board with this FPGA with the same tight design will produce the same heat. You always loose important part in my messages - MiSTer doesn't add more heat to generic work. Load the Linux supplied by Terasic and compare the heating with MiSTer project.
But you don't want (or can't?) understand what i'm telling and try to provide strange proofs.

ijor wrote:Man, please stop being so aggressive. You complain all the time that it is too much work for a single developer. And you are absolutely right about that. You made an awesome job and mostly alone. But you are not helping yourself like that. Being arrogant and aggressive almost at the point of making a fight for any voice of disagreement is, IMHO, not the best way.

Biting the single developer - this is not the best way to help the project. If you disagree with the code then offer your fixes/patches.
It's absolutely non-constructive to argue here. So many smart people around an only one is actually doing.
Currently, you act as user who got the project for free but feeling it's not enough you can get. So, come here to push me work harder. I'm such a lazy and stupid dev who still cannot fit your expectations. You only see my aggression, but don't see you ignorance provoking this aggression. If you expect from me "Thank you for biting for my bad work." "I promise to fix all problems you have - just bite me harder - i like it!" - then you are wrong.
If you are generic user who don't understand in programming, then just accept what you get and follow my recommendations.
If you call yourself as developer, then start to help the project. If you can help but feel this project doesn't worth your time and effort, then simply pass.

I also have many genius ideas in my head - most of them usually break when it comes to implementation. I suggest you to start to implement your genius ideas as well - may be you will have less illusions later ;)

User avatar
Newsdee
Atari God
Atari God
Posts: 1129
Joined: Fri Sep 19, 2014 8:40 am

Re: DE-10 nano temperature issues

Postby Newsdee » Wed Nov 08, 2017 6:34 am

Guys, I don't think anybody was asking for any fixes? We are a small enough community that it pains me to see fighting. All input is useful. I get Sorgelig's frustration in being one of the main contributors and for all we know he is getting a lot of requests offline to fix things, even for stuff he did not write. However I also feel Ijor is just trying to help understand the issue, and I didn't get the vibe he was making any requests or demands. We should value transparency in these projects so that people who want to help know what needs attention, but it doesn't mean we have to fix everything either. I'll buy you both a beer and let's play some games on MiSTer. :)

Sorgelig
Atari God
Atari God
Posts: 1933
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: DE-10 nano temperature issues

Postby Sorgelig » Wed Nov 08, 2017 7:37 am

Newsdee wrote:However I also feel Ijor is just trying to help understand the issue, and I didn't get the vibe he was making any requests or demands.


ijor wrote:Cores should not fail if they meet timing with the slow model. It makes you think that perhaps some cores are not properly constrained.

isn't it a request? Make a proper constrains and stop to complain - what else should i say?
I don't see ANY offer of help. Only complains, complains, complains. Some complainers call themselves as a developers with big knowledge, writing smart words. My explanations are being refused. Trying to find something between my words "do you remember you've said, blah blh blah..." And again, only complains and complains...
No one wants to help. Only get it fully ready. And if not, then act as a teacher pointing where i have to fix.
And then ask "Why you are so aggressive?"

User avatar
Newsdee
Atari God
Atari God
Posts: 1129
Joined: Fri Sep 19, 2014 8:40 am

Re: DE-10 nano temperature issues

Postby Newsdee » Wed Nov 08, 2017 10:41 am

You are not alone in this project. I am a bit late in doing my part but getting there (on the only thing I am capable of, firmware and USB). Others are helping too in the capacity they can. I certainly don't expect you to have to look at everything. :wink:

If a problem comes up, great! I think it's best to document it first and let those who are willing analyze it fully. Only then there is the question of what to do about it. It's also totally fine if nobody is willing to bother fixing it. Nobody is asking anybody to have a perfect device. :P

I have zero clue in what goes in a constraints file, but I understand it as "cores can be optimized further". That feels true (also for MiST) but I imagine it's a realm of quickly diminishing returns (i.e. lots of effort for small improvement). So maybe I'd answer that as saying that it's possible but finding out would need some experimentation they are free to try :)

Sorgelig
Atari God
Atari God
Posts: 1933
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: DE-10 nano temperature issues

Postby Sorgelig » Wed Nov 08, 2017 11:10 am

Newsdee wrote:I have zero clue in what goes in a constraints file, but I understand it as "cores can be optimized further".

This is the one of main files in HDL project. It describes the signals relations, allowed delays and positions between them. It's possible to improve the core stability if you know the critical places. I believe only few people are really masters at this task. Usually HDL projects either have no such files (SDC), or just basic one without tweaking.

User avatar
Newsdee
Atari God
Atari God
Posts: 1129
Joined: Fri Sep 19, 2014 8:40 am

Re: DE-10 nano temperature issues

Postby Newsdee » Wed Nov 08, 2017 11:23 am

Is there any way that people with real hardware (but no HDL knowledge) could help with? Like measure signals with an oscilloscope and document them? Long shot maybe, but I suppose potential testers are in broader supply than devs :)

This is a question for all devs... there are several willing people to help but many struggle getting anywhere useful in HDL. So worth knowing from the experts in which way, if any, some of us could help with development or testing. I tested myself the NES core extensively so if anybody is willing to pick it up I can be your source of info / guinea pig :)

User avatar
alexh
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2651
Joined: Wed Oct 20, 2004 1:52 pm
Location: UK - Oxford
Contact:

Re: DE-10 nano temperature issues

Postby alexh » Wed Nov 08, 2017 11:36 am

Sorgelig wrote:
alexh wrote:https://www.altera.com/en_US/pdfs/literature/an/an734.pdf

Where it says otherwise?

It talks about how the ARM can be instructed to go to WFI/WFE (wait for interrupt) where dynamic power is greatly reduced by putting the processor into standby (sleep).

Sorgelig wrote:Where it tells about cpu idling or dynamic frequency change like cpu in phone for example?

It doesn't. But that is not what I thought you were asking about. "ARM CPU in Cyclone V has no power management. It cannot sleep/idle."

Sorgelig wrote:There is NO power management in its classic form. Document only describes how to reduce power by switching off unneeded components. It's applicable only for bare-metal programming, not for Linux.

I would disagree about terminology. This IS "classic power management". But yes this is not a "governor". AFAIK ARM Linux can use and benefit from the power management available in the Cyclone V. Which version of Linux are you using (I didn't realise MiSTer used Linux at all).

Sorgelig wrote:Instead pointing me to documents, i suggest to build your own version of Linux where you will greatly reduce the heating. Everything is open source, so just take it and improve if you think you know it better.

I don't know, I just believe. I am providing good positive feedback. I am interested and with feedback from you maybe I will take a look and see if I can help. I'll go have a research about which Linux you use and how it is configured for ARM C-States.


Return to “MiSTer”

Who is online

Users browsing this forum: THaase and 2 guests