Linux grub2 /boot partitions strangeness

User discussion and information resource forum for BootIt Bare Metal and BootIt UEFI
Post Reply
eldiener
Posts: 67
Joined: Thu Apr 12, 2012 2:37 am

Linux grub2 /boot partitions strangeness

Post by eldiener »

The issue I will describe here is not necessarily a BIBM issue, but it does show up when I use the Terabyte GRUB2 bootable disk to bootup a Linux OS, so I will ask it here and maybe someone will recognize what might be happening.

I have multiple Linux distros and Windows OSs installed on a computer and BIBM works great. Two of my Linux distros are Mint and Ubuntu. Each one has its own separate /boot partitions and they are, in Linux terms, /dev/sda10 and /dev/sda11 respectively on my first hard disk. I ordinarily have no problems pointing to these /boot partitions and using BIBM to boot into those distros. In my Linux distros I always use UUIDs in the respective /etc/fstabs to denote the actual partitions but of course grub2 itself uses /dev/sdann notation in its grub-install and update-grub commands.

Although the Mint /boot partition is /dev/sda10 and the Ubuntu /boot partition is /dev/sda11, they are actually physically in reverse order on my first hard disk. In other words physically /dev/sda10 comes after /dev/sda11 on the hard disk. At some point in the past they were physically in the normal order but I had to restore backups of these boot partitions from an external hard drive and the physical position of the partitions on the hard drive got reversed as far as the /dev/sda10 and /dev/sda11 notation was concerned. Nonetheless booting into Mint or Ubuntu through BIBM continued to work.

Now for the problem I am seeing. When I am booted into Mint, update software ( I use Synaptic ), and either some part of the grub2 software gets updated or a new kernel gets installed, evidently the 'update-grub' command gets run automatically because I can see the update process regenerating the grub2 menu. This works fine when I reboot into Mint with BIBM but unfortunately when I boot into Ubuntu with BIBM I now see the Mint grub2 menu of items and not the Ubuntu menu of items. What could be happening here ? My original though is that this strictly a Mint problem, in which I case I shouldn't be bothering to ask about it here.

So now I use the Terabyte GRUB2 restore program from a bootable disk to boot into Ubuntu and regenerate the correct Ubuntu grub2 menu, and it works like a charm. But here is the odd thing and the reason I am asking about this on BIBM's online forum here. When I run the Terabyte GRUB2 restore program from a bootable DVD, after setting my partitions for the Ubuntu BIBM menu item beforehand which hides all but the necessary Ubuntu partitions I need, I start by using the "search -f --no-floppy /grub/grub.cfg' from the grub2 prompt. And what is returned to me is both the 'hd0,msdos11' and 'hd0,msdos10' partitions in that order. Of course 'hd0,msdos11' is the one I want and the one I use in the 'configfile' command to reboot into Ubuntu so I can fix the Ubuntu grub2 bootup menu. But why would the 'search' command be finding both the Ubuntu and Mint boot partition cfg files when I have hidden the Mint /boot partition ( along with a bunch of other Linux distro and Windows partitions ) before using the GRUB2 recovery program from my bootable DVD ? So my thinking that this odd behavior has something to do with Mint placing its grub2 menu items on both my Mint and Ubuntu /boot partitions has something to do with this strangeness, so I am asking about this here.

If anybody has any clue or thoughts about what is happening in this situation it would be greatly appreciated.
TeraByte Support
Posts: 3596
Joined: Thu May 05, 2011 10:37 pm

Re: Linux grub2 /boot partitions strangeness

Post by TeraByte Support »

linux in general doesn't adhere to many standard ways of doing things on PC
(some of which you now see in windows), so in this case they just ignore the
file system id and look at a partition no matter what it is. If you use
EMBR in unlimited mode, you can totally hide the other one.


"eldiener" wrote in message news:12633@public.bootitbm...

The issue I will describe here is not necessarily a BIBM issue, but it does
show up when I use the Terabyte GRUB2 bootable disk to bootup a Linux OS, so
I will ask it here and maybe someone will recognize what might be happening.

I have multiple Linux distros and Windows OSs installed on a computer and
BIBM works great. Two of my Linux distros are Mint and Ubuntu. Each one has
its own separate /boot partitions and they are, in Linux terms, /dev/sda10
and /dev/sda11 respectively on my first hard disk. I ordinarily have no
problems pointing to these /boot partitions and using BIBM to boot into
those distros. In my Linux distros I always use UUIDs in the respective
/etc/fstabs to denote the actual partitions but of course grub2 itself uses
/dev/sdann notation in its grub-install and update-grub commands.

Although the Mint /boot partition is /dev/sda10 and the Ubuntu /boot
partition is /dev/sda11, they are actually physically in reverse order on my
first hard disk. In other words physically /dev/sda10 comes after /dev/sda11
on the hard disk. At some point in the past they were physically in the
normal order but I had to restore backups of these boot partitions from an
external hard drive and the physical position of the partitions on the hard
drive got reversed as far as the /dev/sda10 and /dev/sda11 notation was
concerned. Nonetheless booting into Mint or Ubuntu through BIBM continued to
work.

Now for the problem I am seeing. When I am booted into Mint, update software
( I use Synaptic ), and either some part of the grub2 software gets updated
or a new kernel gets installed, evidently the 'update-grub' command gets run
automatically because I can see the update process regenerating the grub2
menu. This works fine when I reboot into Mint with BIBM but unfortunately
when I boot into Ubuntu with BIBM I now see the Mint grub2 menu of items and
not the Ubuntu menu of items. What could be happening here ? My original
though is that this strictly a Mint problem, in which I case I shouldn't be
bothering to ask about it here.

So now I use the Terabyte GRUB2 restore program from a bootable disk to boot
into Ubuntu and regenerate the correct Ubuntu grub2 menu, and it works like
a charm. But here is the odd thing and the reason I am asking about this on
BIBM's online forum here. When I run the Terabyte GRUB2 restore program from
a bootable DVD, after setting my partitions for the Ubuntu BIBM menu item
beforehand which hides all but the necessary Ubuntu partitions I need, I
start by using the "search -f --no-floppy /grub/grub.cfg' from the grub2
prompt. And what is returned to me is both the 'hd0,msdos11' and
'hd0,msdos10' partitions in that order. Of course 'hd0,msdos11' is the one I
want and the one I use in the 'configfile' command to reboot into Ubuntu so
I can fix the Ubuntu grub2 bootup menu. But why would the 'search' command
be finding both the Ubuntu and Mint boot partition cfg files when I have
hidden the Mint /boot partition ( along with a bunch of other Linux distro
and Windows partitions ) before using the GRUB2 recovery program from my
bootable DVD ? So my thinking that this odd behavior has something to do
with Mint placing its grub2 menu items on both my Mint and Ubuntu /boot
partitions has something to do with this strangeness, so I am asking about
this here.

If anybody has any clue or thoughts about what is happening in this
situation it would be greatly appreciated.

eldiener
Posts: 67
Joined: Thu Apr 12, 2012 2:37 am

Re: Linux grub2 /boot partitions strangeness

Post by eldiener »

TeraByte Support wrote:
> linux in general doesn't adhere to many standard ways of doing things on PC
>
> (some of which you now see in windows), so in this case they just ignore
> the
> file system id and look at a partition no matter what it is. If you use
> EMBR in unlimited mode, you can totally hide the other one.

What has this information to do with my OP ?

>
>
> "eldiener" wrote in message news:12633@public.bootitbm...
>
> The issue I will describe here is not necessarily a BIBM issue, but it does
>
> show up when I use the Terabyte GRUB2 bootable disk to bootup a Linux OS,
> so
> I will ask it here and maybe someone will recognize what might be
> happening.
>
> I have multiple Linux distros and Windows OSs installed on a computer and
> BIBM works great. Two of my Linux distros are Mint and Ubuntu. Each one has
>
> its own separate /boot partitions and they are, in Linux terms, /dev/sda10
>
> and /dev/sda11 respectively on my first hard disk. I ordinarily have no
> problems pointing to these /boot partitions and using BIBM to boot into
> those distros. In my Linux distros I always use UUIDs in the respective
> /etc/fstabs to denote the actual partitions but of course grub2 itself uses
>
> /dev/sdann notation in its grub-install and update-grub commands.
>
> Although the Mint /boot partition is /dev/sda10 and the Ubuntu /boot
> partition is /dev/sda11, they are actually physically in reverse order on
> my
> first hard disk. In other words physically /dev/sda10 comes after
> /dev/sda11
> on the hard disk. At some point in the past they were physically in the
> normal order but I had to restore backups of these boot partitions from an
>
> external hard drive and the physical position of the partitions on the hard
>
> drive got reversed as far as the /dev/sda10 and /dev/sda11 notation was
> concerned. Nonetheless booting into Mint or Ubuntu through BIBM continued
> to
> work.
>
> Now for the problem I am seeing. When I am booted into Mint, update
> software
> ( I use Synaptic ), and either some part of the grub2 software gets updated
>
> or a new kernel gets installed, evidently the 'update-grub' command gets
> run
> automatically because I can see the update process regenerating the grub2
> menu. This works fine when I reboot into Mint with BIBM but unfortunately
> when I boot into Ubuntu with BIBM I now see the Mint grub2 menu of items
> and
> not the Ubuntu menu of items. What could be happening here ? My original
> though is that this strictly a Mint problem, in which I case I shouldn't be
>
> bothering to ask about it here.
>
> So now I use the Terabyte GRUB2 restore program from a bootable disk to
> boot
> into Ubuntu and regenerate the correct Ubuntu grub2 menu, and it works like
>
> a charm. But here is the odd thing and the reason I am asking about this on
>
> BIBM's online forum here. When I run the Terabyte GRUB2 restore program
> from
> a bootable DVD, after setting my partitions for the Ubuntu BIBM menu item
> beforehand which hides all but the necessary Ubuntu partitions I need, I
> start by using the "search -f --no-floppy /grub/grub.cfg' from the grub2
> prompt. And what is returned to me is both the 'hd0,msdos11' and
> 'hd0,msdos10' partitions in that order. Of course 'hd0,msdos11' is the one
> I
> want and the one I use in the 'configfile' command to reboot into Ubuntu so
>
> I can fix the Ubuntu grub2 bootup menu. But why would the 'search' command
>
> be finding both the Ubuntu and Mint boot partition cfg files when I have
> hidden the Mint /boot partition ( along with a bunch of other Linux distro
>
> and Windows partitions ) before using the GRUB2 recovery program from my
> bootable DVD ? So my thinking that this odd behavior has something to do
> with Mint placing its grub2 menu items on both my Mint and Ubuntu /boot
> partitions has something to do with this strangeness, so I am asking about
>
> this here.
>
> If anybody has any clue or thoughts about what is happening in this
> situation it would be greatly appreciated.
CyberSimian
Posts: 137
Joined: Wed Aug 10, 2016 12:13 pm

Re: Linux grub2 /boot partitions strangeness

Post by CyberSimian »

eldiener wrote:
> What has this information to do with my OP ?

The practice that I follow is never to allow one operating system to see the partitions occupied by other operating systems. I do this by using EMBR disks instead of MBR disks, and then not including any of the other operating system partitions in the EMBR. In this way, the OS being booted cannot see the other OS partitions, and if it cannot see them, it cannot possibly fiddle with them.

I have a recollection that at one time certain releases of Windows would delete the restore points of other Windows installations, if it could see them in other partitions. I don't understand your setup, but I would speculate that one Linux installation can see the other one, and so can fiddle with it. If this is the case, the solution is to use EMBRs in place of MBRs, and then modify your boot items to include in the boot item only the OS being booted; omit the partitions for the other operating systems.

Of course, this requires you to have your data in separate partitions, so that the data partitions can be included in all boot items (I want this, so I would assume that you do too). Also, to prevent a misguided OS overwriting the apparently unused areas on your disk, you should define some tiny filler partitions so that you can include them in the EMBR to ensure that there are no unused entries in the EMBR. If there are no unused entries, a misguided OS cannot possibly create new partitions for its own purposes (and thereby overwrite the other operating systems on your disk).

-- from CyberSimian in the UK
sigi
Posts: 243
Joined: Sun Jan 15, 2012 1:59 am

Re: Linux grub2 /boot partitions strangeness

Post by sigi »

CyberSimian wrote:
>
> If there are no unused entries, a misguided OS cannot possibly create new partitions for
> its own purposes (and thereby overwrite the other operating systems on your disk).
>
> -- from CyberSimian in the UK


I have a couple of images on my internal HDD's. I could fill the unused slots in the MBR Details with some of those instead of additionally created filler partitions. Do you imply that any filler partitions whatsoever would not be harmed by ill-behaving OS's, even if those were from MS and need to create additional system partitions of their own?
CyberSimian
Posts: 137
Joined: Wed Aug 10, 2016 12:13 pm

Re: Linux grub2 /boot partitions strangeness

Post by CyberSimian »

sigi wrote:
> I have a couple of images on my internal HDD's. I could fill the unused slots in the
> MBR Details with some of those instead of additionally created filler partitions. Do
> you imply that any filler partitions whatsoever would not be harmed by ill-behaving
> OS's, even if those were from MS and need to create additional system partitions of
> their own?

An EMBR has four entries. If you use one entry for the operating system partition, and one for your data partition, that leaves two unused entries. If you left them unused, an operating system could decide to create a new partition (for example, a recovery partition during a large OS update). If there are no unused entries, the operating system cannot do that.

So, what to put in the unused entries in the EMBR? I don't fill the unused entries with other operating systems. Example: if you boot Windows 10, and fill one of the unused entries with Windows 7 (or vice versa), Windows 10 might decide to fiddle with Windows 7. Will it? I don't know, but I prefer not to take the risk.

If you have three data partitions, you can fill the EMBR with those. But if you have only one data partition, I would suggest creating tiny filler partitions at the end of the disk. With cylinder alignment, the smallest partition that you can create is 8 MiB, but with 1 MiB alignment, you can have 1 MiB partitions (so the space "wasted" is minimal). The filler partitions are simply empty formatted FAT16 partitions (too small for FAT32, I think).

When I am installing an operating system, I like to "protect" my data partitions too, so I use three filler partitions on the target install disk, and four filler partitions on other physical disks in the system (if more than one disk is present). Once the install is complete, then I setup the boot items to include the data partitions that I want, plus as many filler partitions as are needed.

-- from CyberSimian in the UK
eldiener
Posts: 67
Joined: Thu Apr 12, 2012 2:37 am

Re: Linux grub2 /boot partitions strangeness

Post by eldiener »

CyberSimian wrote:
> eldiener wrote:
> > What has this information to do with my OP ?
>
> The practice that I follow is never to allow one operating system to see the
> partitions occupied by other operating systems. I do this by using EMBR disks
> instead of MBR disks, and then not including any of the other operating system
> partitions in the EMBR. In this way, the OS being booted cannot see the other OS
> partitions, and if it cannot see them, it cannot possibly fiddle with them.
>
> I have a recollection that at one time certain releases of Windows would delete the
> restore points of other Windows installations, if it could see them in other
> partitions. I don't understand your setup, but I would speculate that one Linux
> installation can see the other one, and so can fiddle with it. If this is the case,
> the solution is to use EMBRs in place of MBRs, and then modify your boot items to
> include in the boot item only the OS being booted; omit the partitions for the other
> operating systems.
>
> Of course, this requires you to have your data in separate partitions, so that the
> data partitions can be included in all boot items (I want this, so I would assume
> that you do too). Also, to prevent a misguided OS overwriting the apparently unused
> areas on your disk, you should define some tiny filler partitions so that you can
> include them in the EMBR to ensure that there are no unused entries in the EMBR. If
> there are no unused entries, a misguided OS cannot possibly create new partitions for
> its own purposes (and thereby overwrite the other operating systems on your disk).
>
> -- from CyberSimian in the UK

I don't use BIBM system of allowing more than 4 primary partitions through EMBR manipulation for various boot items. The simple reason I do not use it, besides never finding a need for it although others might, is that I cannot backup and restore partitions through BIBM because BIBM cannot see my external hard drives while gparted and minitools can. And I need all my partitions to be normally accessed when I use gparted or minitools.

But aside from all this my post tried to make clear that it may not be a bug in Mint that is causing it to overwrite the Ubuntu grub2 menu in the Ubuntu /boot partition but maybe something else. This is evidenced by the grub2 search command in the Terabyte grub2 recover program where, despite "hiding' my Mint /boot partition when I boot from the Terabyte grub2 recovery dvd, the grub2 search command finds the grub.cfg file both in the Ubuntu /boot partition, where I would expect it to find it, but also in the Mint /boot partition which it should not even be seeing.
DrTeeth
Posts: 1289
Joined: Fri Aug 12, 2011 6:58 pm

Re: Linux grub2 /boot partitions strangeness

Post by DrTeeth »

On Sat, 8 Oct 2016 19:10:39 PDT, just as I was about to take a herb,
eldiener disturbed my reverie and wrote:

>I don't use BIBM system of allowing more than 4 primary partitions through
>EMBR manipulation for various boot items.
>The simple reason I do not use it, besides never finding a need for it
>although others might, is that I cannot backup and restore
>partitions through BIBM because BIBM cannot
>see my external hard drives while gparted and minitools can.

I have 4 windows versions and 5 Linux distros installed here without
problem.

*Every* strange and funky post-install problem I have ever encountered
with *any* OS has been down to NOT hiding OSs from each other. A
previous poster has mentioned that you should use > 4 partitions and
EMBR disks and I fully agree. I would also go as far as not adding
other partitions to the MBR and then hiding them pre-install, I do not
even enter them at all pre-install as I have had some OSs being able
to circumvent the hiding - at least by showing confusing info leading
to operator error, a.k.a.a "senior moment".

I would advise you to operate > 4 partitions and invest in the
excellent TBU backup tools for an easier life. FWIW, I discovered the
other day that IfL is 5 times faster than IfD (in BIBM) on my system
in backing up.

Hope this helps.
--
Cheers,

DrT

"If you want to find out what is wrong
with democracy, spend five minutes with
the average voter." - Winston Churchill
George
Posts: 38
Joined: Tue Aug 30, 2011 5:28 pm

Re: Linux grub2 /boot partitions strangeness

Post by George »

A little background from me. I, too, have several Linux distros and one
Windows system installed on my PC.
My Win 10 and two related volumes (in an extended partition) are on one
HD. On another HD, I have the system boot (BIBM) and one Linux system
and a FAT-32 volume which is shared between Windows and Linux. I can
store data on that volume and then read it using another system; my
solution to transfer files between Linux and Windows. My third HD
contains four Linux distros and a /home volume normally mounted to one
distro but I do mount it to other distros to copy source code to the
running system so I can recompile it. Compilers and supporting
libraries do vary between distros.

I limit partitions on all drives and use volumes in extended partitions.
Like you, I can boot from volumes. One of my bootable Linux volumes
is on sdc6.

I reread your OP to try to understand your problem. When you run the
grub2 restore program from the bootable disk, the underlying OS is
running and all partitions/volumes are visible. It's like hiding a
partition/volume before running "sudo blkid" which lists every
partition/volume on every disk that can be seen.

As a test, I hid an OS (Wily) on sdc6 from another (Xenial) using BIBM.
When I booted Xenial and ran "sudo blkid", I could see the hidden
volume's attributes.

I'm not sure where to go from here so I'll leave it up to you. Maybe
grub-update is doing a mount by uuid. That's a bit beyond me for now as
I have other higher priority items to deal with.

George
---
There are 10 kinds of people in the world.
Those who understand binary and
Those who don't.



On 10/7/2016 6:23 AM, eldiener wrote:
> The issue I will describe here is not necessarily a BIBM issue, but it does show up when I use the Terabyte GRUB2 bootable disk to bootup a Linux OS, so I will ask it here and maybe someone will recognize what might be happening.
>
> I have multiple Linux distros and Windows OSs installed on a computer and BIBM works great. Two of my Linux distros are Mint and Ubuntu. Each one has its own separate /boot partitions and they are, in Linux terms, /dev/sda10 and /dev/sda11 respectively on my first hard disk. I ordinarily have no problems pointing to these /boot partitions and using BIBM to boot into those distros. In my Linux distros I always use UUIDs in the respective /etc/fstabs to denote the actual partitions but of course grub2 itself uses /dev/sdann notation in its grub-install and update-grub commands.
>
> Although the Mint /boot partition is /dev/sda10 and the Ubuntu /boot partition is /dev/sda11, they are actually physically in reverse order on my first hard disk. In other words physically /dev/sda10 comes after /dev/sda11 on the hard disk. At some point in the past they were physically in the normal order but I had to restore backups of these boot partitions from an external hard drive and the physical position of the partitions on the hard drive got reversed as far as the /dev/sda10 and /dev/sda11 notation was concerned. Nonetheless booting into Mint or Ubuntu through BIBM continued to work.
>
> Now for the problem I am seeing. When I am booted into Mint, update software ( I use Synaptic ), and either some part of the grub2 software gets updated or a new kernel gets installed, evidently the 'update-grub' command gets run automatically because I can see the update process regenerating the grub2 menu. This works fine when I reboot into Mint with BIBM but unfortunately when I boot into Ubuntu with BIBM I now see the Mint grub2 menu of items and not the Ubuntu menu of items. What could be happening here ? My original though is that this strictly a Mint problem, in which I case I shouldn't be bothering to ask about it here.
>
> So now I use the Terabyte GRUB2 restore program from a bootable disk to boot into Ubuntu and regenerate the correct Ubuntu grub2 menu, and it works like a charm. But here is the odd thing and the reason I am asking about this on BIBM's online forum here. When I run the Terabyte GRUB2 restore program from a bootable DVD, after setting my partitions for the Ubuntu BIBM menu item beforehand which hides all but the necessary Ubuntu partitions I need, I start by using the "search -f --no-floppy /grub/grub.cfg' from the grub2 prompt. And what is returned to me is both the 'hd0,msdos11' and 'hd0,msdos10' partitions in that order. Of course 'hd0,msdos11' is the one I want and the one I use in the 'configfile' command to reboot into Ubuntu so I can fix the Ubuntu grub2 bootup menu. But why would the 'search' command be finding both the Ubuntu and Mint boot partition cfg files when I have hidden the Mint /boot partition ( along with a bunch of other Linux distro and Windows partitions ) before using the GRUB2 recovery program from my bootable DVD ? So my thinking that this odd behavior has something to do with Mint placing its grub2 menu items on both my Mint and Ubuntu /boot partitions has something to do with this strangeness, so I am asking about this here.
>
> If anybody has any clue or thoughts about what is happening in this situation it would be greatly appreciated.
>
>

eldiener
Posts: 67
Joined: Thu Apr 12, 2012 2:37 am

Re: Linux grub2 /boot partitions strangeness

Post by eldiener »

DrTeeth wrote:
> On Sat, 8 Oct 2016 19:10:39 PDT, just as I was about to take a herb,
> eldiener disturbed my reverie and wrote:
>
> >I don't use BIBM system of allowing more than 4 primary partitions through
>
> >EMBR manipulation for various boot items.
> >The simple reason I do not use it, besides never finding a need for it
> >although others might, is that I cannot backup and restore
> >partitions through BIBM because BIBM cannot
> >see my external hard drives while gparted and minitools can.
>
> I have 4 windows versions and 5 Linux distros installed here without
> problem.
>
> *Every* strange and funky post-install problem I have ever encountered
> with *any* OS has been down to NOT hiding OSs from each other. A
> previous poster has mentioned that you should use > 4 partitions and
> EMBR disks and I fully agree. I would also go as far as not adding
> other partitions to the MBR and then hiding them pre-install, I do not
> even enter them at all pre-install as I have had some OSs being able
> to circumvent the hiding - at least by showing confusing info leading
> to operator error, a.k.a.a "senior moment".
>
> I would advise you to operate > 4 partitions and invest in the
> excellent TBU backup tools for an easier life. FWIW, I discovered the
> other day that IfL is 5 times faster than IfD (in BIBM) on my system
> in backing up.

None of the TB tools can see my external drives to which I backup my partitions. So using TB to backup and restore partition data is useless for me. Since I must use other bootable ISOs to do this ( gparted, minitools ) trying to use BIBM with more than 4 primary partitions using EMBR is NOT an option for me.

>
> Hope this helps.
> --
> Cheers,
>
> DrT
Post Reply