How to check Optane SLOG/L2ARC and Swap is working after upgrade TrueNAS 12 to 13

AVSION

Contributor
Joined
Dec 28, 2016
Messages
128
Hi all HNY,

I have Migrated my TrueNAS VM one ESXI back to bare metal and also at the same time upgraded from TrueNAS 12 to TrueNAS CORE 13 U3.1.
I have a feeling that the Optane SLOG/L2ARC and Swap setup is slow or not working. first time i set it up was in 2018 on a FreeNAS version using STUX post. The SLOG/L2ARC and Swap is loading using Init Task:

Code:
swapoff -a ; grep -v -E 'none[[:blank:]]+swap[[:blank:]]' /etc/fstab > /etc/fstab.new && echo "/dev/gptid/6762c737-c0a1-11e8-9f75-0cc47adb3218.eli none swap sw 0 0" >> /etc/fstab.new && mv /etc/fstab.new /etc/fstab ; swapon -a


is that script still current to the latest TrueNAS 13 version?

When i did a fresh install of TrueNAS 13 i chose without swap as i have the Optane, is that correct or related to the issue?

as well under system > advance i get auto complete errors relating storage and Self-Encrypting Drive. i tried to update the settings however every time i reboot it change back, see attached screenshot as well some command related to the Optane SLOG/L2ARC and Swap setup.

I would like to fix the Optane SLOG/L2ARC and Swap setup issues with TrueNAS 13 version and make sure its working properly,

Please let me know if you need more information,

Hope you can help,

Code:
xxxxxxx@truenas[~]# zpool status Tank
  pool: Tank
 state: ONLINE
  scan: scrub repaired 0B in 06:55:08 with 0 errors on Sun Dec  4 06:55:09 2022
config:

    NAME                                            STATE     READ WRITE CKSUM
    Tank                                            ONLINE       0     0     0
      raidz2-0                                      ONLINE       0     0     0
        gptid/b5b40bb4-4e99-11e7-9cba-0cc47adb3218  ONLINE       0     0     0
        gptid/b69faf01-4e99-11e7-9cba-0cc47adb3218  ONLINE       0     0     0
        gptid/b792e7b8-4e99-11e7-9cba-0cc47adb3218  ONLINE       0     0     0
        gptid/b87cfa92-4e99-11e7-9cba-0cc47adb3218  ONLINE       0     0     0
        gptid/b969428a-4e99-11e7-9cba-0cc47adb3218  ONLINE       0     0     0
        gptid/ba5cf8b5-4e99-11e7-9cba-0cc47adb3218  ONLINE       0     0     0
        gptid/bb4ae668-4e99-11e7-9cba-0cc47adb3218  ONLINE       0     0     0
        gptid/bc3d4265-4e99-11e7-9cba-0cc47adb3218  ONLINE       0     0     0
    logs
      gptid/6ff84a55-c0a1-11e8-9f75-0cc47adb3218    ONLINE       0     0     0
    cache
      gptid/76d01ebd-c0a1-11e8-9f75-0cc47adb3218    ONLINE       0     0     0


Code:
xxxxxx@truenas[~]# glabel status | grep nvd0
gptid/6ff84a55-c0a1-11e8-9f75-0cc47adb3218     N/A  nvd0p2
gptid/76d01ebd-c0a1-11e8-9f75-0cc47adb3218     N/A  nvd0p3
gptid/6762c737-c0a1-11e8-9f75-0cc47adb3218     N/A  nvd0p1


Code:
xxxxxx@truenas[~]# swapinfo
Device          1K-blocks     Used    Avail Capacity
/dev/mirror/swap0.eli   2097152        0  2097152     0%
/dev/mirror/swap1.eli   2097152        0  2097152     0%
/dev/mirror/swap2.eli   2097152        0  2097152     0%
/dev/mirror/swap3.eli   2097152        0  2097152     0%
/dev/gptid/6762c737-c0a1-11e8-9  16777216        0 16777216     0%
Total            25165824        0 25165824     0%


Screen Shot 2023-01-10 at 12.56.47.png
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,703
under system > advance i get auto complete errors relating storage and Self-Encrypting Drive. i tried to update the settings however every time i reboot it change back, see attached screenshot as well some command related to the Optane SLOG/L2ARC and Swap setup.
Looks like you use a password manager in your browser... you need to clear those entries or stop the autofill for that site in the password manager.
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,703
I have a feeling that the Optane SLOG/L2ARC and Swap setup is slow or not working.
Check the output from arc_summary

Personally, I wouldn't be messing with swap, just letting it go on the pool disks (and never getting used if your system works properly)
 

AVSION

Contributor
Joined
Dec 28, 2016
Messages
128
Looks like you use a password manager in your browser... you need to clear those entries or stop the autofill for that site in the password manager.
Thank you that worked i manage to save the settings
 

AVSION

Contributor
Joined
Dec 28, 2016
Messages
128
Check the output from arc_summary

Personally, I wouldn't be messing with swap, just letting it go on the pool disks (and never getting used if your system works properly
 

AVSION

Contributor
Joined
Dec 28, 2016
Messages
128
Thank you see summery printout, what do you think is the Optane SLOG/L2ARC and Swap working?

Code:
ZFS Subsystem Report                            Tue Jan 10 22:10:29 2023
FreeBSD 13.1-RELEASE-p2                                    zpl version 5
Machine: truenas.avsion.com.au (amd64)                  spa version 5000

ARC status:                                                      HEALTHY
        Memory throttle count:                                         0

ARC size (current):                                    96.2 %   83.0 GiB
        Target size (adaptive):                        96.4 %   83.2 GiB
        Min size (hard limit):                          3.5 %    3.0 GiB
        Max size (high water):                           28:1   86.3 GiB
        Most Frequently Used (MFU) cache size:          6.6 %    5.5 GiB
        Most Recently Used (MRU) cache size:           93.4 %   77.3 GiB
        Metadata cache size (hard limit):              75.0 %   64.7 GiB
        Metadata cache size (current):                  0.7 %  438.2 MiB
        Dnode cache size (hard limit):                 10.0 %    6.5 GiB
        Dnode cache size (current):                     0.2 %   15.0 MiB

ARC hash breakdown:
        Elements max:                                               1.1M
        Elements current:                              90.2 %       1.0M
        Collisions:                                               313.8k
        Chain max:                                                     4
        Chains:                                                    29.6k

ARC misc:
        Deleted:                                                    4.2M
        Mutex misses:                                                108
        Eviction skips:                                            16.1k
        Eviction skips due to L2 writes:                               0
        L2 cached evictions:                                    14.0 GiB
        L2 eligible evictions:                                   5.9 GiB
        L2 eligible MFU evictions:                     87.1 %    5.2 GiB
        L2 eligible MRU evictions:                     12.9 %  785.9 MiB
        L2 ineligible evictions:                               520.9 GiB

ARC total accesses (hits + misses):                                 1.8G
        Cache hit ratio:                              100.0 %       1.8G
        Cache miss ratio:                             < 0.1 %     364.3k
        Actual hit ratio (MFU + MRU hits):            100.0 %       1.8G
        Data demand efficiency:                        94.5 %       3.9M
        Data prefetch efficiency:                      32.2 %     145.5k

Cache hits by cache type:
        Most frequently used (MFU):                    99.8 %       1.7G
        Most recently used (MRU):                       0.2 %       3.2M
        Most frequently used (MFU) ghost:             < 0.1 %      48.6k
        Most recently used (MRU) ghost:               < 0.1 %       8.7k
        Anonymously used:                             < 0.1 %     113.1k

Cache hits by data type:
        Demand data:                                    0.2 %       3.7M
        Demand prefetch data:                         < 0.1 %      46.9k
        Demand metadata:                               99.8 %       1.7G
        Demand prefetch metadata:                     < 0.1 %     376.9k

Cache misses by data type:
        Demand data:                                   59.9 %     218.3k
        Demand prefetch data:                          27.1 %      98.6k
        Demand metadata:                               12.1 %      44.1k
        Demand prefetch metadata:                       0.9 %       3.4k

DMU prefetch efficiency:                                            3.3M
        Hit ratio:                                     22.9 %     746.8k
        Miss ratio:                                    77.1 %       2.5M

L2ARC status:                                                    HEALTHY
        Low memory aborts:                                             0
        Free on write:                                                 2
        R/W clashes:                                                   0
        Bad checksums:                                                 0
        I/O errors:                                                    0

L2ARC size (adaptive):                                          43.2 GiB
        Compressed:                                    83.0 %   35.9 GiB
        Header size:                                  < 0.1 %  234.8 KiB
        MFU allocated size:                            32.7 %   11.7 GiB
        MRU allocated size:                            66.7 %   23.9 GiB
        Prefetch allocated size:                        0.5 %  195.9 MiB
        Data (buffer content) allocated size:          99.7 %   35.7 GiB
        Metadata (buffer content) allocated size:       0.3 %  113.1 MiB

L2ARC breakdown:                                                  330.1k
        Hit ratio:                                      9.7 %      32.1k
        Miss ratio:                                    90.3 %     298.0k
        Feeds:                                                     83.9k

L2ARC writes:
        Writes sent:                                    100 %      19.4k

L2ARC evicts:
        Lock retries:                                                  0
        Upon reading:                                                  0

Tunables:
        abd_scatter_enabled                                            1
        abd_scatter_min_size                                        4097
        allow_redacted_dataset_mount                                   0
        anon_data_esize                                                0
        anon_metadata_esize                                            0
        anon_size                                                 263168
        arc.average_blocksize                                       8192
        arc.dnode_limit                                                0
        arc.dnode_limit_percent                                       10
        arc.dnode_reduce_percent                                      10
        arc.evict_batch_limit                                         10
        arc.eviction_pct                                             200
        arc.grow_retry                                                 0
        arc.lotsfree_percent                                          10
        arc.max                                              92619000000
        arc.meta_adjust_restarts                                    4096
        arc.meta_limit                                                 0
        arc.meta_limit_percent                                        75
        arc.meta_min                                                   0
        arc.meta_prune                                             10000
        arc.meta_strategy                                              1
        arc.min                                                        0
        arc.min_prefetch_ms                                            0
        arc.min_prescient_prefetch_ms                                  0
        arc.p_dampener_disable                                         1
        arc.p_min_shift                                                0
        arc.pc_percent                                                 0
        arc.prune_task_threads                                         1
        arc.shrink_shift                                               0
        arc.sys_free                                                   0
        arc_free_target                                           521315
        arc_max                                              92619000000
        arc_min                                                        0
        arc_no_grow_shift                                              5
        async_block_max_blocks                      18446744073709551615
        autoimport_disable                                             1
        btree_verify_intensity                                         0
        ccw_retry_interval                                           300
        checksum_events_per_second                                    20
        commit_timeout_pct                                             5
        compressed_arc_enabled                                         1
        condense.indirect_commit_entry_delay_ms                        0
        condense.indirect_obsolete_pct                                25
        condense.indirect_vdevs_enable                                 1
        condense.max_obsolete_bytes                           1073741824
        condense.min_mapping_bytes                                131072
        condense_pct                                                 200
        crypt_sessions                                                 0
        dbgmsg_enable                                                  1
        dbgmsg_maxsize                                           4194304
        dbuf.cache_shift                                               5
        dbuf.metadata_cache_max_bytes               18446744073709551615
        dbuf.metadata_cache_shift                                      6
        dbuf_cache.hiwater_pct                                        10
        dbuf_cache.lowater_pct                                        10
        dbuf_cache.max_bytes                        18446744073709551615
        dbuf_state_index                                               0
        ddt_data_is_special                                            1
        deadman.checktime_ms                                       60000
        deadman.enabled                                                1
        deadman.failmode                                            wait
        deadman.synctime_ms                                       600000
        deadman.ziotime_ms                                        300000
        debug                                                          0
        debugflags                                                     0
        dedup.prefetch                                                 0
        default_bs                                                     9
        default_ibs                                                   15
        delay_min_dirty_percent                                       60
        delay_scale                                               500000
        dirty_data_max                                        4294967296
        dirty_data_max_max                                    4294967296
        dirty_data_max_max_percent                                    25
        dirty_data_max_percent                                        10
        dirty_data_sync_percent                                       20
        disable_ivset_guid_check                                       0
        dmu_object_alloc_chunk_shift                                   7
        dmu_offset_next_sync                                           1
        dmu_prefetch_max                                       134217728
        dtl_sm_blksz                                                4096
        embedded_slog_min_ms                                          64
        flags                                                          0
        fletcher_4_impl [fastest] scalar superscalar superscalar4 sse2 ssse3 avx2
        free_bpobj_enabled                                             1
        free_leak_on_eio                                               0
        free_min_time_ms                                            1000
        history_output_max                                       1048576
        immediate_write_sz                                         32768
        initialize_chunk_size                                    1048576
        initialize_value                            16045690984833335022
        keep_log_spacemaps_at_export                                   0
        l2arc.exclude_special                                          0
        l2arc.feed_again                                               1
        l2arc.feed_min_ms                                            200
        l2arc.feed_secs                                                1
        l2arc.headroom                                                 2
        l2arc.headroom_boost                                         200
        l2arc.meta_percent                                            33
        l2arc.mfuonly                                                  0
        l2arc.noprefetch                                               0
        l2arc.norw                                                     0
        l2arc.rebuild_blocks_min_l2size                       1073741824
        l2arc.rebuild_enabled                                          0
        l2arc.trim_ahead                                               0
        l2arc.write_boost                                       40000000
        l2arc.write_max                                         10000000
        l2arc_feed_again                                               1
        l2arc_feed_min_ms                                            200
        l2arc_feed_secs                                                1
        l2arc_headroom                                                 2
        l2arc_noprefetch                                               0
        l2arc_norw                                                     0
        l2arc_write_boost                                       40000000
        l2arc_write_max                                         10000000
        l2c_only_size                                                  0
        livelist.condense.new_alloc                                    0
        livelist.condense.sync_cancel                                  0
        livelist.condense.sync_pause                                   0
        livelist.condense.zthr_cancel                                  0
        livelist.condense.zthr_pause                                   0
        livelist.max_entries                                      500000
        livelist.min_percent_shared                                   75
        lua.max_instrlimit                                     100000000
        lua.max_memlimit                                       104857600
        max_async_dedup_frees                                     100000
        max_auto_ashift                                               14
        max_dataset_nesting                                           50
        max_log_walking                                                5
        max_logsm_summary_length                                      10
        max_missing_tvds                                               0
        max_missing_tvds_cachefile                                     2
        max_missing_tvds_scan                                          0
        max_nvlist_src_size                                            0
        max_recordsize                                           1048576
        metaslab.aliquot                                         1048576
        metaslab.bias_enabled                                          1
        metaslab.debug_load                                            0
        metaslab.debug_unload                                          0
        metaslab.df_alloc_threshold                               131072
        metaslab.df_free_pct                                           4
        metaslab.df_max_search                                  16777216
        metaslab.df_use_largest_segment                                0
        metaslab.find_max_tries                                      100
        metaslab.force_ganging                                  16777217
        metaslab.fragmentation_factor_enabled                          1
        metaslab.fragmentation_threshold                              70
        metaslab.lba_weighting_enabled                                 1
        metaslab.load_pct                                             50
        metaslab.max_size_cache_sec                                 3600
        metaslab.mem_limit                                            25
        metaslab.preload_enabled                                       1
        metaslab.preload_limit                                        10
        metaslab.segment_weight_enabled                                1
        metaslab.sm_blksz_no_log                                   16384
        metaslab.sm_blksz_with_log                                131072
        metaslab.switch_threshold                                      2
        metaslab.try_hard_before_gang                                  0
        metaslab.unload_delay                                         32
        metaslab.unload_delay_ms                                  600000
        mfu_data_esize                                        3044419072
        mfu_ghost_data_esize                                 14973300224
        mfu_ghost_metadata_esize                               113598464
        mfu_ghost_size                                       15086898688
        mfu_metadata_esize                                       7204352
        mfu_size                                              5902172160
        mg.fragmentation_threshold                                    95
        mg.noalloc_threshold                                           0
        min_auto_ashift                                                9
        min_metaslabs_to_flush                                         1
        mru_data_esize                                       81883939840
        mru_ghost_data_esize                                  5706481664
        mru_ghost_metadata_esize                               415357440
        mru_ghost_size                                        6121839104
        mru_metadata_esize                                      24153088
        mru_size                                             82960787968
        multihost.fail_intervals                                      10
        multihost.history                                              0
        multihost.import_intervals                                    20
        multihost.interval                                          1000
        multilist_num_sublists                                         0
        no_scrub_io                                                    0
        no_scrub_prefetch                                              0
        nocacheflush                                                   0
        nopwrite_enabled                                               1
        obsolete_min_time_ms                                         500
        pd_bytes_max                                            52428800
        per_txg_dirty_frees_percent                                    5
        prefetch.array_rd_sz                                     1048576
        prefetch.disable                                               0
        prefetch.max_distance                                   67108864
        prefetch.max_idistance                                  67108864
        prefetch.max_sec_reap                                          2
        prefetch.max_streams                                           8
        prefetch.min_distance                                    4194304
        prefetch.min_sec_reap                                          1
        read_history                                                   0
        read_history_hits                                              0
        rebuild_max_segment                                      1048576
        rebuild_scrub_enabled                                          1
        rebuild_vdev_limit                                      33554432
        reconstruct.indirect_combinations_max                       4096
        recover                                                        0
        recv.queue_ff                                                 20
        recv.queue_length                                       16777216
        recv.write_batch_size                                    1048576
        removal_suspend_progress                                       0
        remove_max_segment                                      16777216
        resilver_disable_defer                                         0
        resilver_min_time_ms                                        3000
        scan_blkstats                                                  0
        scan_checkpoint_intval                                      7200
        scan_fill_weight                                               3
        scan_ignore_errors                                             0
        scan_issue_strategy                                            0
        scan_legacy                                                    0
        scan_max_ext_gap                                         2097152
        scan_mem_lim_fact                                             20
        scan_mem_lim_soft_fact                                        20
        scan_strict_mem_lim                                            0
        scan_suspend_progress                                          0
        scan_vdev_limit                                          4194304
        scrub_min_time_ms                                           1000
        send.corrupt_data                                              0
        send.no_prefetch_queue_ff                                     20
        send.no_prefetch_queue_length                            1048576
        send.override_estimate_recordsize                              0
        send.queue_ff                                                 20
        send.queue_length                                       16777216
        send.unmodified_spill_blocks                                   1
        send_holes_without_birth_time                                  1
        slow_io_events_per_second                                     20
        spa.asize_inflation                                           24
        spa.discard_memory_limit                                16777216
        spa.load_print_vdev_tree                                       0
        spa.load_verify_data                                           1
        spa.load_verify_metadata                                       1
        spa.load_verify_shift                                          4
        spa.slop_shift                                                 5
        space_map_ibs                                                 14
        special_class_metadata_reserve_pct                            25
        standard_sm_blksz                                         131072
        super_owner                                                    0
        sync_pass_deferred_free                                        2
        sync_pass_dont_compress                                        8
        sync_pass_rewrite                                              2
        sync_taskq_batch_pct                                          75
        top_maxinflight                                             1000
        traverse_indirect_prefetch_limit                              32
        trim.extent_bytes_max                                  134217728
        trim.extent_bytes_min                                      32768
        trim.metaslab_skip                                             0
        trim.queue_limit                                              10
        trim.txg_batch                                                32
        txg.history                                                  100
        txg.timeout                                                    5
        unflushed_log_block_max                                   131072
        unflushed_log_block_min                                     1000
        unflushed_log_block_pct                                      400
        unflushed_log_txg_max                                       1000
        unflushed_max_mem_amt                                 1073741824
        unflushed_max_mem_ppm                                       1000
        user_indirect_is_special                                       1
        validate_skip                                                  0
        vdev.aggregate_trim                                            0
        vdev.aggregation_limit                                   1048576
        vdev.aggregation_limit_non_rotating                       131072
        vdev.async_read_max_active                                     3
        vdev.async_read_min_active                                     1
        vdev.async_write_active_max_dirty_percent                     60
        vdev.async_write_active_min_dirty_percent                     30
        vdev.async_write_max_active                                    5
        vdev.async_write_min_active                                    1
        vdev.bio_delete_disable                                        0
        vdev.bio_flush_disable                                         0
        vdev.cache_bshift                                             16
        vdev.cache_max                                             16384
        vdev.cache_size                                                0
        vdev.def_queue_depth                                          32
        vdev.default_ms_count                                        200
        vdev.default_ms_shift                                         29
        vdev.file.logical_ashift                                       9
        vdev.file.physical_ashift                                      9
        vdev.initializing_max_active                                   1
        vdev.initializing_min_active                                   1
        vdev.max_active                                             1000
        vdev.max_auto_ashift                                          14
        vdev.min_auto_ashift                                           9
        vdev.min_ms_count                                             16
        vdev.mirror.non_rotating_inc                                   0
        vdev.mirror.non_rotating_seek_inc                              1
        vdev.mirror.rotating_inc                                       0
        vdev.mirror.rotating_seek_inc                                  5
        vdev.mirror.rotating_seek_offset                         1048576
        vdev.ms_count_limit                                       131072
        vdev.nia_credit                                                5
        vdev.nia_delay                                                 5
        vdev.queue_depth_pct                                        1000
        vdev.read_gap_limit                                        32768
        vdev.rebuild_max_active                                        3
        vdev.rebuild_min_active                                        1
        vdev.removal_ignore_errors                                     0
        vdev.removal_max_active                                        2
        vdev.removal_max_span                                      32768
        vdev.removal_min_active                                        1
        vdev.removal_suspend_progress                                  0
        vdev.remove_max_segment                                 16777216
        vdev.scrub_max_active                                          3
        vdev.scrub_min_active                                          1
        vdev.sync_read_max_active                                     10
        vdev.sync_read_min_active                                     10
        vdev.sync_write_max_active                                    10
        vdev.sync_write_min_active                                    10
        vdev.trim_max_active                                           2
        vdev.trim_min_active                                           1
        vdev.validate_skip                                             0
        vdev.write_gap_limit                                        4096
        version.acl                                                    1
        version.ioctl                                                 15
        version.module                         v2022101100-zfs_2a0dae1b7
        version.spa                                                 5000
        version.zpl                                                    5
        vnops.read_chunk_size                                    1048576
        vol.mode                                                       2
        vol.recursive                                                  0
        vol.unmap_enabled                                              1
        wrlog_data_max                                        8589934592
        xattr_compat                                                   1
        zap_iterate_prefetch                                           1
        zevent.len_max                                               512
        zevent.retain_expire_secs                                    900
        zevent.retain_max                                           2000
        zfetch.max_distance                                     67108864
        zfetch.max_idistance                                    67108864
        zil.clean_taskq_maxalloc                                 1048576
        zil.clean_taskq_minalloc                                    1024
        zil.clean_taskq_nthr_pct                                     100
        zil.maxblocksize                                          131072
        zil.nocacheflush                                               0
        zil.replay_disable                                             0
        zil.slog_bulk                                             786432
        zio.deadman_log_all                                            0
        zio.dva_throttle_enabled                                       1
        zio.exclude_metadata                                           0
        zio.requeue_io_start_cut_in_line                               1
        zio.slow_io_ms                                             30000
        zio.taskq_batch_pct                                           80
        zio.taskq_batch_tpq                                            0
        zio.use_uma                                                    1

VDEV cache disabled, skipping section

ZIL committed transactions:                                        13.0M
        Commit requests:                                            7.3M
        Flushes to stable storage:                                  7.0M
        Transactions to SLOG storage pool:          632.3 GiB       9.1M
        Transactions to non-SLOG storage pool:        6.7 MiB        251
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,703
what do you think is the Optane SLOG/L2ARC and Swap working?
Your ARC is catching a lot of requests for metadata and you have a really high hit ratio of ~100%

L2ARC is catching about 10% of the requests it's being asked for, so not providing a lot of help, but maybe a little. I would personally see it as mostly a waste of TBW on that device.

Your SLOG is doing its work (see the second last line in the report).

That report doesn't cover swap... but I expect SWAP is never used, so anything you're seeing is only related to its setup.
 

AVSION

Contributor
Joined
Dec 28, 2016
Messages
128
Thank for your insight,

Regarding ARC you mean i used all my memory capacity? i have total of 96GB ATM i'm running smb share and other basic ssh services no VMs or something special, what is a normal/recommended/Target ARC hit ratio? how do i find the issue and improve it?

L2ARC - So the Cashing is actually working from/using the Optane or different drive? i set the optane L2ARC to 20g, how do i find the issue and improve it?

what is recommended for both of the above?

attached hew screenshots

Thank you

Screen Shot 2023-01-11 at 10.27.10.png


Code:
last pid: 30970;  load averages:  0.30,  0.13,  0.09                                                                   up 1+13:39:40  11:19:57
53 processes:  1 running, 52 sleeping
CPU:  0.0% user,  0.0% nice,  0.1% system,  0.0% interrupt, 99.9% idle
Mem: 38M Active, 978M Inact, 587M Laundry, 88G Wired, 4124M Free
ARC: 83G Total, 8549M MFU, 74G MRU, 2105K Anon, 238M Header, 33M Other
     80G Compressed, 103G Uncompressed, 1.28:1 Ratio
Swap: 24G Total, 24G Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
21937 root         11  20    0   124M    67M nanslp   6   3:06   1.74% collectd
 3632 root          8  20    0    57M    13M select  23   4:30   0.65% rrdcached
22045 root          4  25    0    84M    54M select   8  29:13   0.52% python3.9
 2011 root         44  20    0   617M   347M kqread   9  10:59   0.23% python3.9
30970 root          1  20    0    14M  4444K CPU0     0   0:00   0.10% top
23019 root          1  20    0   154M   124M kqread  19   0:12   0.08% smbd
22038 root          1  20    0    28M    13M select  24   0:11   0.03% snmpd
 3968 avahi         1  20    0    14M  4932K select  23   0:31   0.01% avahi-daemon
 3903 root          1  20    0    45M    23M kqread  22   0:02   0.00% nmbd
 3600 ntpd          1  20    0    21M  7128K select  27   0:05   0.00% ntpd
 3647 root          1  20    0    19M  8496K select   5   0:03   0.00% proftpd
13016 root          1  20    0    20M  9120K select  26   0:01   0.00% sshd
 3757 www           1  20    0    36M  9100K kqread  25   0:03   0.00% nginx
 3717 root          1  20    0    52M    26M kqread  10   0:02   0.00% winbindd
 3728 root          1  20    0    53M    26M kqread  24   0:00   0.00% winbindd
 3724 root          1  20    0    92M    66M kqread  22   0:00   0.00% winbindd
 2352 root          3  20    0   234M   178M usem    16   1:19   0.00% python3.9
 2353 root          3  20    0   238M   178M usem    12   1:18   0.00% python3.9
 2421 root          3  25    0   234M   178M usem    23   1:16   0.00% python3.9
 2106 root          3  20    0   255M   178M usem    18   1:15   0.00% python3.9
 2354 root          3  20    0   238M   178M piperd  24   1:14   0.00% python3.9
 3556 root          1 -52   r0    13M    13M nanslp   7   0:15   0.00% watchdogd
 2133 root          5  20    0    92M    58M usem    20   0:04   0.00% python3.9
 3693 root          1  20    0   141M   113M kqread   4   0:04   0.00% smbd
 3714 root          1  20    0   138M   110M kqread  27   0:02   0.00% smbd
22006 root          1  20    0    17M  6496K nanslp   0   0:01   0.00% smartd
 3534 root          2  20    0    39M    11M kqread  22   0:01   0.00% syslog-ng
 2134 root          1  20    0    65M    50M zevent  25   0:01   0.00% python3.9
 1977 root          1  20    0    11M  2012K select  22   0:00   0.00% devd
 3739 root          1  20    0    13M  2812K nanslp   0   0:00   0.00% cron
 3855 root          1  20    0    61M    42M ttyin    3   0:00   0.00% python3.9
 3863 root          1  52    0    61M    42M ttyin    8   0:00   0.00% python3.9
 3993 root          1  20    0    40M    26M kqread  24   0:00   0.00% python3.9
 3716 root          1  20    0   139M   110M kqread   9   0:00   0.00% smbd
 2012 root          1  20    0    23M    12M piperd  23   0:00   0.00% python3.9
13018 root          1  20    0    15M  4816K pause    5   0:00   0.00% zsh
 1993 root          1  20    0    19M  6192K select   1   0:00   0.00% zfsd
 3640 root          1  20    0    20M  8416K select  13   0:00   0.00% sshd
 3858 root          1  52    0    13M  2560K ttyin   23   0:00   0.00% getty
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,703
No swap used is great. That's exactly what you want.

Using all your RAM is great too (most of that will be ARC... ZFS cache)

ARC hit ratio refers to how often something the system is asked to provide from disk can instead be provided from RAM (ARC). A high ratio is desirable/expected and yours is close to perfect (not surprising with a good amount of RAM).

L2ARC is on your NVME drive (from your pool status and glabel status above). As I said, it's not doing much (since you have plenty of RAM/ARC) with a hit ratio of under 10%... I would recommend to just remove it.

Other than removing the L2ARC, I see nothing that needs improvement (at least from what you shared so far).
 

AVSION

Contributor
Joined
Dec 28, 2016
Messages
128
Excellent, i thought it was the other way around that the system run out of ram :smile: thank you.

There are 3 partitions on the Optane SWAP 16G, SLOG 20G and L2ARC is the rest 225G, few questions:
  • if i remove the L2ARC does it work on the ram or it use ARC only, otherwise how does it work?
  • if i remove the L2ARC do i relocate or keep the SLOG and the SWAP?
  • or you mean to keep the SLOG and SWAP on the Optane and use the 225G for another high speed use?
  • if so what is my options or what you recommend using of the storage on the Optane?
  • any good resource how to remove the L2ARC?
Thank you
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,703
if i remove the L2ARC does it work on the ram or it use ARC only, otherwise how does it work?
L2ARC is an "extension" of ARC, allowing things that would be ejected from ARC due to RAM constraints instead be written to a fast disk... if you don't have L2ARC, you just have ARC (which is always there).

if i remove the L2ARC do i relocate or keep the SLOG and the SWAP?
Your choice. Removing L2ARC from a pool does nothing to the other items.

or you mean to keep the SLOG and SWAP on the Optane and use the 225G for another high speed use?
You could do that (keeping in mind it's not recommended to share a SLOG device for anything else).

if so what is my options or what you recommend using of the storage on the Optane?
Continue using it as SLOG (if you think you'll continue to need SLOG (VMs or block storage needing sync writes mostly)... but seriously consider your pool design if that's the case, since RAIDZ is a terrible choice for block storage: https://www.truenas.com/community/threads/the-path-to-success-for-block-storage.81165/

any good resource how to remove the L2ARC?
zpool remove Tank gptid/76d01ebd-c0a1-11e8-9f75-0cc47adb3218
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,703
Depending on how critical your VMs are, you could run them on the optane (and probably no longer need SLOG... and I would eliminate the Swap from it too), with replication (every few hours or every day, depending on how much you can afford to lose) to the RAIDZ pool cover for failure of the Optane
 
Top