# IO error: No space left on device

**URL:** <https://forum.xx.network/t/io-error-no-space-left-on-device/6830>\
**Category:** How-To and Common Errors\
**Created:** [July 21, 2022, 3:43am UTC](https://forum.xx.network/t/io-error-no-space-left-on-device/6830 "2022-07-21T03:43:45Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Keith](https://forum.xx.network/user_avatar/forum.xx.network/keith/32/159_2.png) [@Keith](https://forum.xx.network/u/Keith)\
**Post date:** [July 21, 2022, 3:43am UTC](https://forum.xx.network/t/io-error-no-space-left-on-device/6830/1 "2022-07-21T03:43:45Z")

</div>

As time and blocks pass, the disk space required to store the blockchain data grows.

As of the writing of this post, approximately 82GB is needed …

```auto
$ du -d 1 -h /opt/xxnetwork | grep db
82G /opt/xxnetwork/db

```

In recent days, I have on a few occasions been contacted by node operators whose xxnetwork-chain process crashes and the chain.log contains messages with `No space left on device`

The log message may vary …

```auto
Error: Service(Client(Backend("Io error: No space left on device (os error 28)")))

GRANDPA voter error: Client(DatabaseError(DatabaseError(Custom { kind: Other, error: Error { message: "IO error: No space left on device: While appending to file: /opt/xxnetwork/db/chains/xxnetwork/db/full/10105292.log: No space left on device" } })))

DatabaseError(DatabaseError(Custom { kind: Other, error: Error { message: "IO error: No space left on device: While appending to file: /opt/xxnetwork/db/chains/xxnetwork/db/full/6431858.log: No space left on device" } }))

```

… but the one thing in common is “No space left on device.”

Assuming the size of the computer’s storage device meets the [Hardware Requirements](https://handbook.xx.network/specifications) :

- Node – 1TB
- Gateway – 500GB

… if `xxnetwork-chain` repeatedly crashes with one of the previously mentioned `No space left on device` messages, then it likely means the full available capacity of the storage device was not partitioned and formatted during the installation of the OS.

For example, nvme0n1, a 1TB (931.5G available) disk only had 100GB allocated to the Logical Volume named `ubuntu--vg-ubuntu--lv` and mounted on `/`

```auto
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
nvme0n1 259:0 0 931.5G 0 disk
├─nvme0n1p1 259:1 0 1.1G 0 part /boot/efi
├─nvme0n1p2 259:2 0 1.5G 0 part /boot
└─nvme0n1p3 259:3 0 929G 0 part
  └─ubuntu--vg-ubuntu--lv 253:0 0 100G 0 lvm /

```

As a result, you can see 98% of `ubuntu--vg-ubuntu--lv` mounted on `/` is used.

```auto
$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/ubuntu--vg-ubuntu--lv 98G 91G 2.8G 98% /
/dev/nvme0n1p2 1.5G 367M 1016M 27% /boot
/dev/nvme0n1p1 1.1G 5.3M 1.1G 1% /boot/efi

```

To rectify the situation, the Logical Volume can be extended to use the full available capacity of the partition. There are multiple ways to accomplish this so it is best you refer to general Ubuntu support via the internet which will provide you plenty of resources …

Search for keywords such as `ubuntu extend lvm partition` using [DuckDuckGo](https://duckduckgo.com/?q=ubuntu+extend+lvm+root+partition) or [Google](https://www.google.com/search?q=ubuntu+extend+lvm+root+partition&oq=ubuntu+expand+lv).

Once the issue of the disk being full is rectified, I would suggest you have a look at [ERROR or OFFLINE Nodes](https://forum.xx.network/t/error-or-offline-nodes/6523) to ensure all xx network related processes are functioning properly.

Additional notes:

- A full disk can cause other dependent services like PostgreSQL to crash. You might need to restart the postgresql service or just reboot the computer to make sure **everything** is running correctly.
- A full disk may also result in having to re-sync the chain due to database corruption. But, you will want to **double check before** you take steps required to fix [DB has been previously marked as corrupted, attempting repair](https://forum.xx.network/t/db-has-been-previously-marked-as-corrupted-attempting-repair/6660)

And as always, if you’re still having issues please feel free to contact support at nodes@xx.network or via the [xx network Discord channel](https://discord.xx.network)

Thanks for running a node!  
Keith

---

<div class="post-metadata">

**Author:** ![Eyez](https://forum.xx.network/user_avatar/forum.xx.network/eyez/32/264_2.png) [@Eyez](https://forum.xx.network/u/Eyez)\
**Post date:** [July 23, 2022, 8:40pm UTC](https://forum.xx.network/t/io-error-no-space-left-on-device/6830/2 "2022-07-23T20:40:38Z")

</div>

Ty Keith, just extended mine. It was 67% full.

This guide is good

> **[Ubuntu: Extend your default LVM space - Packet Pushers](https://packetpushers.net/ubuntu-extend-your-default-lvm-space/)**
>
> POV: You’re a sysadmin who set up a one-off Linux machine for an app you needed, and now it’s out of disk space. You originally spun up a VM, installed a recent Ubuntu OS, and just hit Next, Next, Finish through the guided install. Linux is not your...

/Simon

---

<div class="post-metadata">

**Author:** ![vipul](https://forum.xx.network/letter_avatar_proxy/v4/letter/v/cdc98d/32.png) [@vipul](https://forum.xx.network/u/vipul)\
**Post date:** [January 8, 2023, 6:54pm UTC](https://forum.xx.network/t/io-error-no-space-left-on-device/6830/3 "2023-01-08T18:54:59Z")

</div>

I also did the space expansion. However, the DB was still full so ended up deleting DB and then changing session keys followed by resyncing chain

---

<div class="post-metadata">

**Author:** ![AlexGas](https://forum.xx.network/user_avatar/forum.xx.network/alexgas/32/867_2.png) [@AlexGas](https://forum.xx.network/u/AlexGas)\
**Post date:** [March 2, 2023, 9:20am UTC](https://forum.xx.network/t/io-error-no-space-left-on-device/6830/4 "2023-03-02T09:20:42Z")

</div>

I also had such a problem. Thanks Keith, we fixed it!
